==================== ISSUES SUMMARY ==================== ========== ODH/RHOAI ISSUES SUMMARY (status, events, highlights only) ========== 173m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-w7ps9 MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 127m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 126m Warning Unhealthy pod/ogx-k8s-operator-controller-manager-67866ffc79-w7ps9 Readiness probe failed: Get "http://10.131.3.224:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 126m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-r6dfz MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 126m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-glbln MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 126m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f764xpql MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 126m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-2xmtz MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 126m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 126m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 126m Warning FailedMount pod/model-serving-api-78bbbb9bf8-bxmjn MountVolume.SetUp failed for volume "tls-certs" : secret "model-serving-api-tls" not found 126m Warning FailedMount pod/llmisvc-controller-manager-586c85c465-rj86k MountVolume.SetUp failed for volume "cert" : secret "llmisvc-webhook-server-cert" not found 125m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 125m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-glbln" 125m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-ztpsf" 125m Warning FailedCreate replicaset/ogx-k8s-operator-controller-manager-67866ffc79 Error creating: pods "ogx-k8s-operator-controller-manager-67866ffc79-" is forbidden: error looking up service account redhat-ods-applications/ogx-k8s-operator-controller-manager: serviceaccount "ogx-k8s-operator-controller-manager" not found 125m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-v4g6q MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 125m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-cwnbf MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 125m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Failed to pull image "registry.redhat.io/rhoai/odh-dashboard-rhel9@sha256:a1a2f0e908ead8d61c07b1fb49f81a60a8f11127aa11a9c8671ba7d04bdef3dc": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/9d/9d8d66840ef8ce081c72b2a2cb3b41a2d4097f1636c9406979c2b3dce3008da8?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260707%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260707T054755Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=0c6409c87fd7d949ad221987b7bcf0e247d68d17b247c7eecbdf5bdddea6377a®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-dashboard-rhel9&akamai_signature=exp=1783404175~hmac=6d1303656401f3475aaf3278143edd3c29339091336541fe246cf859f60d0957": EOF 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Failed to pull image "registry.redhat.io/rhoai/odh-mod-arch-model-registry-rhel9@sha256:1ce99d94d4af0a582e3a98a4e6a7758652a98c2e4dd6fff2380cfdd925a6f5cf": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/a9/a9d40ceb865f72b9ecca3cc5dca6cac11a6e344d4273c7d2f67ca4d4e1556d9c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260707%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260707T054756Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=13c15838d17b440fe23eb2afa63dd1036d7c3ff46406365506205d8d0b15a3a0®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-mod-arch-model-registry-rhel9&akamai_signature=exp=1783404176~hmac=088a5593cb07cd1a0162eb9afb48501dea56cf5262a32ff1911f7e7dc8d816cc": EOF 125m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-v4g6q Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-v4g6q Failed to pull image "registry.redhat.io/rhoai/odh-mod-arch-autorag-rhel9@sha256:ca5b5b8b793dd3203383da034de17111914c3bc4378254ad320ecd45cc44668c": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/ed/edb44819fd83987e87fe3c3937e9c409af0d976f99683f31b22328474706ca25?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260707%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260707T054803Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=a6c5b55d762e9909558783848ca677f3c13dc67f5c7d3f3c009cfd4d42956631®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-mod-arch-autorag-rhel9&akamai_signature=exp=1783404183~hmac=70dd668e499359dd6b75ce5153e1055bf04c138bce73f55781d409a6822a5984": EOF 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ImagePullBackOff 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ImagePullBackOff 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-v4g6q Error: ImagePullBackOff 125m Warning FailedMount pod/kuberay-operator-6b449f854b-s5x6d MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 124m Warning Unhealthy pod/rhods-dashboard-6b5cc9785c-v4g6q Readiness probe failed: Get "https://10.130.3.205:8444/healthz": dial tcp 10.130.3.205:8444: connect: connection refused 124m Warning FailedMount pod/spark-pi-d9696hr88e3s7b173360-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-e28e209f3b1f7a66-conf-map" not found 122m Warning FailedMount pod/scheduled-spark-pi-d9696ub88e3s7b173370-1783403415000651360-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-50794e9f3b20b148-conf-map" not found 122m Warning FailedMount pod/scheduled-spark-pi-d9696ub88e3s7b173370-1783403415000651360-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d9696ub88e3s7b173370-1783403415000651360-12a43b9f3b20a081-driver-podspec-conf-map" not found 122m Warning FailedMount pod/spark-pi-6beb569f3b20d2c5-exec-1 MountVolume.SetUp failed for volume "spark-conf-volume-exec" : configmap "spark-exec-5ed82b9f3b20dbc9-conf-map" not found 122m Warning Unhealthy pod/rhods-dashboard-6b5cc9785c-4cv2g Readiness probe failed: Get "https://10.130.3.211:8444/healthz": dial tcp 10.130.3.211:8444: connect: connection refused 115m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-7fk7q MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 115m Warning InvalidConfiguration servicemonitor/workload-variant-autoscaler-controller-manager-metrics-monitor ServiceMonitor workload-variant-autoscaler-controller-manager-metrics-monitor was rejected due to invalid configuration: failed to get authorization token of type "": unable to get secret "workload-variant-autoscaler-metrics-reader-token": secrets "workload-variant-autoscaler-metrics-reader-token" not found 115m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 112m Warning Unhealthy pod/workload-variant-autoscaler-controller-manager-56865c9848-gr9z7 Readiness probe failed: Get "http://10.130.3.215:8081/readyz": dial tcp 10.130.3.215:8081: connect: connection refused 109m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 109m Warning FailedMount pod/odh-model-controller-54bf759f64-8f7rq MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 109m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 108m Warning FailedMount pod/maas-api-84c4b8dcb6-n5czz MountVolume.SetUp failed for volume "maas-api-tls" : secret "maas-api-serving-cert" not found 108m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 107m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 107m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 107m Warning FailedCreate replicaset/mlflow-operator-controller-manager-8646f94cf4 Error creating: pods "mlflow-operator-controller-manager-8646f94cf4-" is forbidden: error looking up service account redhat-ods-applications/mlflow-operator-controller-manager: serviceaccount "mlflow-operator-controller-manager" not found 107m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 105m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 105m Warning FailedMount pod/odh-model-controller-54bf759f64-ffdgw MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 105m Warning FailedMount pod/model-serving-api-78bbbb9bf8-jpjp9 MountVolume.SetUp failed for volume "tls-certs" : secret "model-serving-api-tls" not found 105m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 104m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 104m Warning Unhealthy pod/maas-controller-656b6867cb-lgtvk Readiness probe failed: HTTP probe failed with statuscode: 500 102m Warning BackOff pod/maas-controller-656b6867cb-lgtvk Back-off restarting failed container manager in pod maas-controller-656b6867cb-lgtvk_redhat-ods-applications(4b2b890c-ac6b-4347-9db8-d02b99fd44a3) 97m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 96m Warning FailedMount pod/odh-model-controller-54bf759f64-cp8tr MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 96m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 96m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 95m Warning FailedToUpdateEndpoint endpoints/llmisvc-webhook-server-service Failed to update endpoint redhat-ods-applications/llmisvc-webhook-server-service: Operation cannot be fulfilled on endpoints "llmisvc-webhook-server-service": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/redhat-ods-applications/llmisvc-webhook-server-service, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f42baaa0-f852-4846-bb1d-1799a0b72f69, UID in object meta: 95m Warning FailedToUpdateEndpointSlices service/kserve-controller-manager-metrics-service Error updating Endpoint Slices for Service redhat-ods-applications/kserve-controller-manager-metrics-service: failed to update kserve-controller-manager-metrics-service-dn5q7 EndpointSlice for Service redhat-ods-applications/kserve-controller-manager-metrics-service: endpointslices.discovery.k8s.io "kserve-controller-manager-metrics-service-dn5q7" not found 95m Warning FailedMount pod/odh-model-controller-54bf759f64-fvd4t MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 95m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 93m Warning FailedMount pod/mlflow-operator-controller-manager-8646f94cf4-5n5c2 MountVolume.SetUp failed for volume "metrics-cert" : secret "controller-manager-metrics-tls" not found 93m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-h7dk4 MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 93m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f76tthhm MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 93m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 93m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-6fjms MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 93m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-zn2c4 MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 93m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 93m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 93m Warning BackOff pod/trustyai-service-operator-controller-manager-6f7f88c874-vmnkf Back-off restarting failed container manager in pod trustyai-service-operator-controller-manager-6f7f88c874-vmnkf_redhat-ods-applications(ebcac290-d6ef-4077-984f-f5f07dff1fe9) 93m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-z9h9d MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-92bq7 MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-z9h9d MountVolume.SetUp failed for volume "modules-sa-token" : references non-existent secret key: token 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-92bq7 MountVolume.SetUp failed for volume "modules-sa-token" : references non-existent secret key: token 93m Warning FailedMount pod/kuberay-operator-6b449f854b-jk5h9 MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 89m Warning Unhealthy pod/mlflow-668464596d-pv868 Readiness probe failed: Get "https://10.131.2.33:8443/mlflow/health": dial tcp 10.131.2.33:8443: connect: connection refused 86m Warning Unhealthy pod/mlflow-58f77fb56c-z2fjq Readiness probe failed: Get "https://10.131.2.35:8443/mlflow/health": dial tcp 10.131.2.35:8443: connect: connection refused 7m41s Warning FailedMount pod/spark-pi-d96at3b88e3s7b359mhg-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-009f8d9f3b8a06ee-conf-map" not found 7m41s Warning FailedMount pod/spark-pi-d96at3b88e3s7b359mhg-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "spark-pi-d96at3b88e3s7b359mhg-098a439f3b89f624-driver-podspec-conf-map" not found 6m22s Warning FailedMount pod/scheduled-spark-pi-d96atg388e3s7b359mi0-1783410398000727474-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d96atg388e3s7b359mi0-1783410398000727474-91293f9f3b8b2dde-driver-podspec-conf-map" not found 6m22s Warning FailedMount pod/scheduled-spark-pi-d96atg388e3s7b359mi0-1783410398000727474-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-5b24549f3b8b3f76-conf-map" not found 3m20s Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 3m20s Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 3m18s Warning FailedToRetrieveImagePullSecret pod/maas-controller-656b6867cb-lgtvk Unable to retrieve some image pull secrets (maas-controller-dockercfg-xfjxt); attempting to pull the image may not succeed. 76m Warning Deleting modelregistry/e2e-test-registry Custom Resource e2e-test-registry is being deleted from the namespace rhoai-model-registries 70m Warning Deleting modelregistry/test-model-registry-662874 Custom Resource test-model-registry-662874 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/e2e-test-registry Custom Resource e2e-test-registry is being deleted from the namespace rhoai-model-registries 166m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 75m Warning Deleting modelregistry/default-db-registry-1783406077384 Custom Resource default-db-registry-1783406077384 is being deleted from the namespace rhoai-model-registries 75m Warning Deleting modelregistry/default-db-registry-1783406077384 Custom Resource default-db-registry-1783406077384 is being deleted from the namespace rhoai-model-registries 167m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 166m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 70m Warning Deleting modelregistry/test-model-registry-662874 Custom Resource test-model-registry-662874 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/postgres-registry-1783406077384 Custom Resource postgres-registry-1783406077384 is being deleted from the namespace rhoai-model-registries 166m Warning FailedMount pod/model-registry0-849dbccdd5-4m2xw MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-registry0-kube-rbac-proxy-config" not found 127m Warning FailedToUpdateEndpointSlices service/model-catalog Error updating Endpoint Slices for Service rhoai-model-registries/model-catalog: failed to update model-catalog-b9qpq EndpointSlice for Service rhoai-model-registries/model-catalog: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "model-catalog-b9qpq": StorageError: invalid object, Code: 4, Key: /kubernetes.io/endpointslices/rhoai-model-registries/model-catalog-b9qpq, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 014b344a-dbfe-435a-9325-e70e8fb2e2d8, UID in object meta: 126m Warning FailedMount pod/model-catalog-7cc8755dc9-c9ntl MountVolume.SetUp failed for volume "kube-rbac-proxy-tls" : secret "model-catalog-kube-rbac-proxy" not found 126m Warning FailedMount pod/model-catalog-7cc8755dc9-c9ntl MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 126m Warning Unhealthy pod/model-catalog-7cc8755dc9-c9ntl Readiness probe failed: dial tcp 10.128.2.170:8080: connect: connection refused 125m Warning FailedToUpdateEndpointSlices service/model-catalog Error updating Endpoint Slices for Service rhoai-model-registries/model-catalog: failed to update model-catalog-rlg55 EndpointSlice for Service rhoai-model-registries/model-catalog: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "model-catalog-rlg55": StorageError: invalid object, Code: 4, Key: /kubernetes.io/endpointslices/rhoai-model-registries/model-catalog-rlg55, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 311661a9-050a-4438-8b48-a307f5f54928, UID in object meta: 125m Warning FailedToUpdateEndpointSlices service/model-catalog-postgres Error updating Endpoint Slices for Service rhoai-model-registries/model-catalog-postgres: failed to update model-catalog-postgres-48pzp EndpointSlice for Service rhoai-model-registries/model-catalog-postgres: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "model-catalog-postgres-48pzp": StorageError: invalid object, Code: 4, Key: /kubernetes.io/endpointslices/rhoai-model-registries/model-catalog-postgres-48pzp, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 9c66614a-4571-46a9-8eb2-feafd2196344, UID in object meta: 115m Warning FailedMount pod/model-catalog-7cc8755dc9-blkbw MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 115m Warning Unhealthy pod/model-catalog-7cc8755dc9-blkbw Readiness probe failed: dial tcp 10.131.1.16:8080: connect: connection refused 93m Warning FailedMount pod/model-catalog-7cc8755dc9-wdzcm MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 93m Warning Unhealthy pod/model-catalog-7cc8755dc9-wdzcm Readiness probe failed: dial tcp 10.129.2.151:8080: connect: connection refused 78m Warning Unhealthy pod/mysql-db-registry-1783406077384-594fc44d89-88vh4 Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure.... 77m Warning FailedMount pod/e2e-test-registry-57cb7c6bb9-lk6vc MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "e2e-test-registry-kube-rbac-proxy-config" not found 76m Warning FailedMount pod/postgres-registry-1783406077384-987f85b98-796nv MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "postgres-registry-1783406077384-kube-rbac-proxy-config" not found 75m Warning Unhealthy pod/default-db-registry-1783406077384-849897cddd-zbb4p Startup probe failed: HTTP probe failed with statuscode: 503 75m Warning Unhealthy pod/default-db-registry-1783406077384-849897cddd-zbb4p Startup probe failed: Get "http://10.129.2.168:8080/readyz/isDirty": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 75m Warning Unhealthy pod/default-db-registry-1783406077384-postgres-7f4d674d67-8wbx7 Readiness probe errored: rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1 74m Warning Unhealthy pod/model-registry-db-662874-7956bc68d9-d4c8l Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure.... 74m Warning FailedMount pod/test-model-registry-662874-748fdd9884-fx2hz MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "test-model-registry-662874-kube-rbac-proxy-config" not found 82m Warning FailedMount pod/jupyter-nb-ldap-2dadmin1-0 MountVolume.SetUp failed for volume "kube-rbac-proxy-tls-certificates" : secret "jupyter-nb-ldap-2dadmin1-kube-rbac-proxy-tls" not found 82m Warning FailedMount notebook/jupyter-nb-ldap-2dadmin1 Reissued from pod/jupyter-nb-ldap-2dadmin1-0: MountVolume.SetUp failed for volume "kube-rbac-proxy-tls-certificates" : secret "jupyter-nb-ldap-2dadmin1-kube-rbac-proxy-tls" not found 74m Warning Unhealthy pod/jupyter-nb-ldap-2dadmin1-0 Readiness probe failed: Get "https://10.128.2.185:8444/healthz": dial tcp 10.128.2.185:8444: connect: connection refused 74m Warning Unhealthy pod/jupyter-nb-ldap-2dadmin1-0 Readiness probe failed: Get "http://10.128.2.185:8888/notebook/rhods-notebooks/jupyter-nb-ldap-2dadmin1/api": dial tcp 10.128.2.185:8888: connect: connection refused 74m Warning Unhealthy notebook/jupyter-nb-ldap-2dadmin1 Reissued from pod/jupyter-nb-ldap-2dadmin1-0: Readiness probe failed: Get "https://10.128.2.185:8444/healthz": dial tcp 10.128.2.185:8444: connect: connection refused 74m Warning Unhealthy notebook/jupyter-nb-ldap-2dadmin1 Reissued from pod/jupyter-nb-ldap-2dadmin1-0: Readiness probe failed: Get "http://10.128.2.185:8888/notebook/rhods-notebooks/jupyter-nb-ldap-2dadmin1/api": dial tcp 10.128.2.185:8888: connect: connection refused {"level":"error","ts":"2026-07-07T06:40:43Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:41:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:42:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:43:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:44:04Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:44:55Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:45:25Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:46:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:46:32Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:47:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:47:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:48:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:49:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:50:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:50:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:51:33Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:52:17Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:52:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:53:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:54:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:54:59Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:55:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:56:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:57:19Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:58:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:58:44Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:59:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:00:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:00:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:01:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:02:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:03:26Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:04:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:04:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:05:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:06:29Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:07:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:07:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:08:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:09:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:10:26Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:10:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:11:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:12:32Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:13:16Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:14:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:15:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:15:46Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:16:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:17:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:17:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:18:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:19:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:19:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:20:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:21:22Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:22:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:22:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:23:43Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:24:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:25:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:25:55Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:26:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:27:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:28:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:29:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:29:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:30:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:31:39Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:32:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:32:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:33:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:34:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:35:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:36:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:37:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:38:00Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:38:58Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:39:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:40:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:41:25Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:42:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:42:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:43:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:44:44Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:45:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:46:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:46:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:47:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:48:17Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:49:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:50:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:51:00Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:51:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:52:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:53:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} --------------------------------------------------------------------------------------------------- ==================== STATUS REPORT ==================== ========== ODH/RHOAI STATUS REPORT (2026-07-07T07:53:09Z) ========== --- OpenShift / OCP version --- Client Version: 4.20.8 Kustomize Version: v5.6.0 Server Version: 4.19.18 Kubernetes Version: v1.32.9 --- Namespaces --- namespace/redhat-ods-applications namespace/redhat-ods-monitoring namespace/redhat-ods-operator namespace/rhoai-model-registries namespace/rhods-notebooks --- Subscriptions (redhat-ods-operator) --- NAME PACKAGE SOURCE CHANNEL rhods-operator rhods-operator rhoai-catalog-dev beta Subscription conditions (unhealthy): CatalogSourcesUnhealthy: status=False reason=AllCatalogSourcesHealthy message=all available catalogsources are healthy --- ClusterServiceVersions (redhat-ods-operator) --- NAME DISPLAY VERSION REPLACES PHASE authorino-operator.v1.4.0 Authorino Operator 1.4.0 authorino-operator.v1.3.1 Succeeded cert-manager-operator.v1.19.0 cert-manager Operator for Red Hat OpenShift 1.19.0 cert-manager-operator.v1.18.1 Succeeded custom-metrics-autoscaler.v2.19.0-1 Custom Metrics Autoscaler 2.19.0-1 custom-metrics-autoscaler.v2.18.1-2 Succeeded dns-operator.v1.4.0 DNS Operator 1.4.0 dns-operator.v1.3.1 Succeeded kueue-operator.v1.3.1 Red Hat build of Kueue 1.3.1 Succeeded limitador-operator.v1.4.0 Limitador Operator 1.4.0 limitador-operator.v1.3.1 Succeeded rhcl-operator.v1.4.0 Red Hat Connectivity Link 1.4.0 rhcl-operator.v1.3.4 Succeeded rhods-operator.3.5.0-ea.2 Red Hat OpenShift AI 3.5.0-ea.2 Succeeded servicemeshoperator3.v3.3.4 Red Hat OpenShift Service Mesh 3 3.3.4 servicemeshoperator3.v3.1.0 Succeeded --- CSVs not in Succeeded phase --- (all CSVs report phase Succeeded) --- InstallPlans (redhat-ods-operator, not Complete) --- (no non-Complete InstallPlans) --- DataScienceCluster --- NAME READY REASON default-dsc True DSC Ready summary: True reason= DSC component states: aipipelines: managementState=Managed dashboard: managementState=Managed feastoperator: managementState=Managed kserve: managementState=Managed kueue: managementState=Unmanaged llamastackoperator: managementState=n/a mlflowoperator: managementState=Managed modelregistry: managementState=Managed ogx: managementState=Managed ray: managementState=Managed sparkoperator: managementState=Managed trainer: managementState=Managed trainingoperator: managementState=Managed trustyai: managementState=Managed workbenches: managementState=Managed DSC conditions: Ready: status=True reason= message= ProvisioningSucceeded: status=True reason= message= ComponentsReady: status=True reason= message= AIPipelinesReady: status=True reason= message= DashboardReady: status=True reason= message= FeastOperatorReady: status=True reason= message= KserveLLMInferenceServiceDependencies: status=True reason= message= KserveLLMInferenceServiceWideEPDependencies: status=True reason= message= KserveReady: status=True reason= message= KueueReady: status=True reason= message= MLflowOperatorReady: status=True reason= message= MaaSPrerequisitesAvailable: status=False reason=MaaSPrerequisitesNotMet message=maas-default-gateway is missing required annotations: security.opendatahub.io/authorino-tls-bootstrap="true" ModelControllerReady: status=True reason= message= ModelRegistryReady: status=True reason= message= ModelsAsServiceReady: status=True reason=Reconciled message=MaaS platform manifests applied and maas-api deployment is available OGXReady: status=True reason= message= RayReady: status=True reason= message= SparkOperatorReady: status=True reason= message= TrainerReady: status=True reason= message= TrainingOperatorReady: status=True reason= message= TrustyAIReady: status=True reason= message= WorkbenchesReady: status=True reason= message= --- DSCInitialization --- NAME AGE PHASE CREATED AT default-dsci 7d18h Ready 2026-06-29T13:43:18Z DSCInitialization conditions: ReconcileComplete: status=True reason=ReconcileCompleted message=Reconcile completed successfully Available: status=True reason=ReconcileCompleted message=Reconcile completed successfully Progressing: status=False reason=ReconcileCompleted message=Reconcile completed successfully Degraded: status=False reason=ReconcileCompleted message=Reconcile completed successfully Upgradeable: status=True reason=ReconcileCompleted message=Reconcile completed successfully MonitoringReady: status=True reason=Ready message=Monitoring stack is initialized Ready: status=True reason= message= ProvisioningSucceeded: status=True reason= message= MonitoringStackAvailable: status=False reason=MetricsNotConfigured message=Metrics not configured in DSCI CR TempoAvailable: status=False reason=TracesNotConfigured message=Traces not configured in DSCI CR OpenTelemetryCollectorAvailable: status=False reason=MetricsNotConfiguredAndTracesNotConfigured message=Metrics not configured in DSCI CR Traces not configured in DSCI CR AlertingAvailable: status=False reason=AlertingNotConfigured message=Alerting not configured in DSCI CR ThanosQuerierAvailable: status=False reason=MetricsNotConfigured message=Metrics not configured in DSCI CR PersesAvailable: status=False reason=MetricsNotConfiguredAndTracesNotConfigured message=Perses requires at least Metrics or Traces to be configured NodeMetricsEndpointAvailable: status=False reason=MetricsNotConfigured message=Metrics not configured in DSCI CR --- Component CRs (Ready) --- NAME READY REASON URL dashboard.components.platform.opendatahub.io/default-dashboard True NAME READY REASON datasciencepipelines.components.platform.opendatahub.io/default-datasciencepipelines True NAME READY REASON feastoperator.components.platform.opendatahub.io/default-feastoperator True NAME READY REASON kserve.components.platform.opendatahub.io/default-kserve True NAME READY REASON kueue.components.platform.opendatahub.io/default-kueue True NAME READY REASON mlflowoperator.components.platform.opendatahub.io/default-mlflowoperator True NAME READY REASON URI modelcontroller.components.platform.opendatahub.io/default-modelcontroller True NAME READY REASON modelregistry.components.platform.opendatahub.io/default-modelregistry True NAME READY REASON modelsasservice.components.platform.opendatahub.io/default-modelsasservice True NAME READY REASON ogx.components.platform.opendatahub.io/default-ogx True NAME READY REASON ray.components.platform.opendatahub.io/default-ray True NAME READY REASON sparkoperator.components.platform.opendatahub.io/default-sparkoperator True NAME READY REASON trainer.components.platform.opendatahub.io/default-trainer True NAME READY REASON trainingoperator.components.platform.opendatahub.io/default-trainingoperator True NAME READY REASON trustyai.components.platform.opendatahub.io/default-trustyai True NAME READY REASON workbenches.components.platform.opendatahub.io/default-workbenches True --- Deployments not fully ready (ODH/RHOAI namespaces) --- (all deployments report ready replicas == desired) --- Pods not Running/Succeeded (ODH/RHOAI namespaces) --- (all pods Running or Succeeded) --------------------------------------------------------------------------------------------------- ==================== EVENTS ==================== ========== ODH/RHOAI EVENTS (Warning / Error / Failed) ========== --- namespace/redhat-ods-applications --- LAST SEEN TYPE REASON OBJECT MESSAGE 173m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-w7ps9 MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 127m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 126m Warning Unhealthy pod/ogx-k8s-operator-controller-manager-67866ffc79-w7ps9 Readiness probe failed: Get "http://10.131.3.224:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 126m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-r6dfz MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 126m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-glbln MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 126m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f764xpql MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 126m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-2xmtz MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 126m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 126m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 126m Warning FailedMount pod/model-serving-api-78bbbb9bf8-bxmjn MountVolume.SetUp failed for volume "tls-certs" : secret "model-serving-api-tls" not found 126m Warning FailedMount pod/llmisvc-controller-manager-586c85c465-rj86k MountVolume.SetUp failed for volume "cert" : secret "llmisvc-webhook-server-cert" not found 125m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 125m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-glbln" 125m Warning CalculateExpectedPodCountFailed poddisruptionbudget/ogx-k8s-operator-controller-manager-pdb Failed to calculate the number of expected pods: found no controllers for pod "ogx-k8s-operator-controller-manager-67866ffc79-ztpsf" 125m Warning FailedCreate replicaset/ogx-k8s-operator-controller-manager-67866ffc79 Error creating: pods "ogx-k8s-operator-controller-manager-67866ffc79-" is forbidden: error looking up service account redhat-ods-applications/ogx-k8s-operator-controller-manager: serviceaccount "ogx-k8s-operator-controller-manager" not found 125m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-v4g6q MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 125m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-cwnbf MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 125m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Failed to pull image "registry.redhat.io/rhoai/odh-dashboard-rhel9@sha256:a1a2f0e908ead8d61c07b1fb49f81a60a8f11127aa11a9c8671ba7d04bdef3dc": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/9d/9d8d66840ef8ce081c72b2a2cb3b41a2d4097f1636c9406979c2b3dce3008da8?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260707%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260707T054755Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=0c6409c87fd7d949ad221987b7bcf0e247d68d17b247c7eecbdf5bdddea6377a®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-dashboard-rhel9&akamai_signature=exp=1783404175~hmac=6d1303656401f3475aaf3278143edd3c29339091336541fe246cf859f60d0957": EOF 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Failed to pull image "registry.redhat.io/rhoai/odh-mod-arch-model-registry-rhel9@sha256:1ce99d94d4af0a582e3a98a4e6a7758652a98c2e4dd6fff2380cfdd925a6f5cf": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/a9/a9d40ceb865f72b9ecca3cc5dca6cac11a6e344d4273c7d2f67ca4d4e1556d9c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260707%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260707T054756Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=13c15838d17b440fe23eb2afa63dd1036d7c3ff46406365506205d8d0b15a3a0®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-mod-arch-model-registry-rhel9&akamai_signature=exp=1783404176~hmac=088a5593cb07cd1a0162eb9afb48501dea56cf5262a32ff1911f7e7dc8d816cc": EOF 125m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-v4g6q Error: ErrImagePull 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-v4g6q Failed to pull image "registry.redhat.io/rhoai/odh-mod-arch-autorag-rhel9@sha256:ca5b5b8b793dd3203383da034de17111914c3bc4378254ad320ecd45cc44668c": copying system image from manifest list: parsing image configuration: Get "https://cdn01.quay.io/quayio-production-s3/sha256/ed/edb44819fd83987e87fe3c3937e9c409af0d976f99683f31b22328474706ca25?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260707%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260707T054803Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=a6c5b55d762e9909558783848ca677f3c13dc67f5c7d3f3c009cfd4d42956631®ion=us-east-1&namespace=rhoai&username=rhoai+rhoai_devops_konfux_bot&repo_name=odh-mod-arch-autorag-rhel9&akamai_signature=exp=1783404183~hmac=70dd668e499359dd6b75ce5153e1055bf04c138bce73f55781d409a6822a5984": EOF 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ImagePullBackOff 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-cwnbf Error: ImagePullBackOff 125m Warning Failed pod/rhods-dashboard-6b5cc9785c-v4g6q Error: ImagePullBackOff 125m Warning FailedMount pod/kuberay-operator-6b449f854b-s5x6d MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 124m Warning Unhealthy pod/rhods-dashboard-6b5cc9785c-v4g6q Readiness probe failed: Get "https://10.130.3.205:8444/healthz": dial tcp 10.130.3.205:8444: connect: connection refused 124m Warning FailedMount pod/spark-pi-d9696hr88e3s7b173360-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-e28e209f3b1f7a66-conf-map" not found 122m Warning FailedMount pod/scheduled-spark-pi-d9696ub88e3s7b173370-1783403415000651360-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-50794e9f3b20b148-conf-map" not found 122m Warning FailedMount pod/scheduled-spark-pi-d9696ub88e3s7b173370-1783403415000651360-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d9696ub88e3s7b173370-1783403415000651360-12a43b9f3b20a081-driver-podspec-conf-map" not found 122m Warning FailedMount pod/spark-pi-6beb569f3b20d2c5-exec-1 MountVolume.SetUp failed for volume "spark-conf-volume-exec" : configmap "spark-exec-5ed82b9f3b20dbc9-conf-map" not found 122m Warning Unhealthy pod/rhods-dashboard-6b5cc9785c-4cv2g Readiness probe failed: Get "https://10.130.3.211:8444/healthz": dial tcp 10.130.3.211:8444: connect: connection refused 115m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-7fk7q MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 115m Warning InvalidConfiguration servicemonitor/workload-variant-autoscaler-controller-manager-metrics-monitor ServiceMonitor workload-variant-autoscaler-controller-manager-metrics-monitor was rejected due to invalid configuration: failed to get authorization token of type "": unable to get secret "workload-variant-autoscaler-metrics-reader-token": secrets "workload-variant-autoscaler-metrics-reader-token" not found 115m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 112m Warning Unhealthy pod/workload-variant-autoscaler-controller-manager-56865c9848-gr9z7 Readiness probe failed: Get "http://10.130.3.215:8081/readyz": dial tcp 10.130.3.215:8081: connect: connection refused 109m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 109m Warning FailedMount pod/odh-model-controller-54bf759f64-8f7rq MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 109m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 108m Warning FailedMount pod/maas-api-84c4b8dcb6-n5czz MountVolume.SetUp failed for volume "maas-api-tls" : secret "maas-api-serving-cert" not found 108m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 107m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 107m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 107m Warning FailedCreate replicaset/mlflow-operator-controller-manager-8646f94cf4 Error creating: pods "mlflow-operator-controller-manager-8646f94cf4-" is forbidden: error looking up service account redhat-ods-applications/mlflow-operator-controller-manager: serviceaccount "mlflow-operator-controller-manager" not found 107m Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 105m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 105m Warning FailedMount pod/odh-model-controller-54bf759f64-ffdgw MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 105m Warning FailedMount pod/model-serving-api-78bbbb9bf8-jpjp9 MountVolume.SetUp failed for volume "tls-certs" : secret "model-serving-api-tls" not found 105m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 104m Warning FailedCreate replicaset/maas-api-84c4b8dcb6 Error creating: pods "maas-api-84c4b8dcb6-" is forbidden: error looking up service account redhat-ods-applications/maas-api: serviceaccount "maas-api" not found 104m Warning Unhealthy pod/maas-controller-656b6867cb-lgtvk Readiness probe failed: HTTP probe failed with statuscode: 500 102m Warning BackOff pod/maas-controller-656b6867cb-lgtvk Back-off restarting failed container manager in pod maas-controller-656b6867cb-lgtvk_redhat-ods-applications(4b2b890c-ac6b-4347-9db8-d02b99fd44a3) 97m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 96m Warning FailedMount pod/odh-model-controller-54bf759f64-cp8tr MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 96m Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 96m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 95m Warning FailedToUpdateEndpoint endpoints/llmisvc-webhook-server-service Failed to update endpoint redhat-ods-applications/llmisvc-webhook-server-service: Operation cannot be fulfilled on endpoints "llmisvc-webhook-server-service": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/redhat-ods-applications/llmisvc-webhook-server-service, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f42baaa0-f852-4846-bb1d-1799a0b72f69, UID in object meta: 95m Warning FailedToUpdateEndpointSlices service/kserve-controller-manager-metrics-service Error updating Endpoint Slices for Service redhat-ods-applications/kserve-controller-manager-metrics-service: failed to update kserve-controller-manager-metrics-service-dn5q7 EndpointSlice for Service redhat-ods-applications/kserve-controller-manager-metrics-service: endpointslices.discovery.k8s.io "kserve-controller-manager-metrics-service-dn5q7" not found 95m Warning FailedMount pod/odh-model-controller-54bf759f64-fvd4t MountVolume.SetUp failed for volume "cert" : secret "odh-model-controller-webhook-cert" not found 95m Warning FailedCreate replicaset/model-serving-api-78bbbb9bf8 Error creating: pods "model-serving-api-78bbbb9bf8-" is forbidden: error looking up service account redhat-ods-applications/model-serving-api: serviceaccount "model-serving-api" not found 93m Warning FailedMount pod/mlflow-operator-controller-manager-8646f94cf4-5n5c2 MountVolume.SetUp failed for volume "metrics-cert" : secret "controller-manager-metrics-tls" not found 93m Warning FailedMount pod/model-registry-operator-controller-manager-5f68f4fd76-h7dk4 MountVolume.SetUp failed for volume "cert" : secret "model-registry-operator-webhook-service" not found 93m Warning FailedMount pod/data-science-pipelines-operator-controller-manager-7fc7f76tthhm MountVolume.SetUp failed for volume "config" : configmap "data-science-pipelines-operator-dspo-config" not found 93m Warning FailedCreate replicaset/trustyai-service-operator-controller-manager-6f7f88c874 Error creating: pods "trustyai-service-operator-controller-manager-6f7f88c874-" is forbidden: error looking up service account redhat-ods-applications/trustyai-service-operator-controller-manager: serviceaccount "trustyai-service-operator-controller-manager" not found 93m Warning FailedMount pod/ogx-k8s-operator-controller-manager-67866ffc79-6fjms MountVolume.SetUp failed for volume "cert" : secret "ogx-k8s-operator-webhook-cert" not found 93m Warning FailedMount pod/odh-notebook-controller-manager-7f6b58f576-zn2c4 MountVolume.SetUp failed for volume "webhook-cert" : secret "odh-notebook-controller-webhook-cert" not found 93m Warning FailedCreate replicaset/spark-operator-controller-57c5867998 Error creating: pods "spark-operator-controller-57c5867998-" is forbidden: error looking up service account redhat-ods-applications/spark-operator-controller: serviceaccount "spark-operator-controller" not found 93m Warning FailedCreate replicaset/notebook-controller-deployment-5d7d79cf5b Error creating: pods "notebook-controller-deployment-5d7d79cf5b-" is forbidden: error looking up service account redhat-ods-applications/notebook-controller-service-account: serviceaccount "notebook-controller-service-account" not found 93m Warning BackOff pod/trustyai-service-operator-controller-manager-6f7f88c874-vmnkf Back-off restarting failed container manager in pod trustyai-service-operator-controller-manager-6f7f88c874-vmnkf_redhat-ods-applications(ebcac290-d6ef-4077-984f-f5f07dff1fe9) 93m Warning FailedCreate replicaset/rhods-dashboard-6b5cc9785c Error creating: pods "rhods-dashboard-6b5cc9785c-" is forbidden: error looking up service account redhat-ods-applications/rhods-dashboard: serviceaccount "rhods-dashboard" not found 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-z9h9d MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-92bq7 MountVolume.SetUp failed for volume "modules-sa-token" : secret "odh-dashboard-modules-token" not found 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-z9h9d MountVolume.SetUp failed for volume "modules-sa-token" : references non-existent secret key: token 93m Warning FailedMount pod/rhods-dashboard-6b5cc9785c-92bq7 MountVolume.SetUp failed for volume "modules-sa-token" : references non-existent secret key: token 93m Warning FailedMount pod/kuberay-operator-6b449f854b-jk5h9 MountVolume.SetUp failed for volume "cert" : secret "kuberay-webhook-server-cert" not found 89m Warning Unhealthy pod/mlflow-668464596d-pv868 Readiness probe failed: Get "https://10.131.2.33:8443/mlflow/health": dial tcp 10.131.2.33:8443: connect: connection refused 86m Warning Unhealthy pod/mlflow-58f77fb56c-z2fjq Readiness probe failed: Get "https://10.131.2.35:8443/mlflow/health": dial tcp 10.131.2.35:8443: connect: connection refused 7m41s Warning FailedMount pod/spark-pi-d96at3b88e3s7b359mhg-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-009f8d9f3b8a06ee-conf-map" not found 7m41s Warning FailedMount pod/spark-pi-d96at3b88e3s7b359mhg-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "spark-pi-d96at3b88e3s7b359mhg-098a439f3b89f624-driver-podspec-conf-map" not found 6m22s Warning FailedMount pod/scheduled-spark-pi-d96atg388e3s7b359mi0-1783410398000727474-driver MountVolume.SetUp failed for volume "pod-template-volume" : configmap "scheduled-spark-pi-d96atg388e3s7b359mi0-1783410398000727474-91293f9f3b8b2dde-driver-podspec-conf-map" not found 6m22s Warning FailedMount pod/scheduled-spark-pi-d96atg388e3s7b359mi0-1783410398000727474-driver MountVolume.SetUp failed for volume "spark-conf-volume-driver" : configmap "spark-drv-5b24549f3b8b3f76-conf-map" not found 3m20s Warning InvalidConfiguration servicemonitor/controller-manager-metrics-monitor ServiceMonitor controller-manager-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 3m20s Warning InvalidConfiguration servicemonitor/odh-model-controller-metrics-monitor ServiceMonitor odh-model-controller-metrics-monitor was rejected due to invalid configuration: it accesses file system via bearer token file which Prometheus specification prohibits 3m18s Warning FailedToRetrieveImagePullSecret pod/maas-controller-656b6867cb-lgtvk Unable to retrieve some image pull secrets (maas-controller-dockercfg-xfjxt); attempting to pull the image may not succeed. --- namespace/redhat-ods-monitoring --- (no events) --- namespace/redhat-ods-operator --- (no events) --- namespace/rhoai-model-registries --- LAST SEEN TYPE REASON OBJECT MESSAGE 76m Warning Deleting modelregistry/e2e-test-registry Custom Resource e2e-test-registry is being deleted from the namespace rhoai-model-registries 70m Warning Deleting modelregistry/test-model-registry-662874 Custom Resource test-model-registry-662874 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/e2e-test-registry Custom Resource e2e-test-registry is being deleted from the namespace rhoai-model-registries 166m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 75m Warning Deleting modelregistry/default-db-registry-1783406077384 Custom Resource default-db-registry-1783406077384 is being deleted from the namespace rhoai-model-registries 75m Warning Deleting modelregistry/default-db-registry-1783406077384 Custom Resource default-db-registry-1783406077384 is being deleted from the namespace rhoai-model-registries 167m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 166m Warning Deleting modelregistry/model-registry0 Custom Resource model-registry0 is being deleted from the namespace rhoai-model-registries 70m Warning Deleting modelregistry/test-model-registry-662874 Custom Resource test-model-registry-662874 is being deleted from the namespace rhoai-model-registries 76m Warning Deleting modelregistry/postgres-registry-1783406077384 Custom Resource postgres-registry-1783406077384 is being deleted from the namespace rhoai-model-registries 166m Warning FailedMount pod/model-registry0-849dbccdd5-4m2xw MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-registry0-kube-rbac-proxy-config" not found 127m Warning FailedToUpdateEndpointSlices service/model-catalog Error updating Endpoint Slices for Service rhoai-model-registries/model-catalog: failed to update model-catalog-b9qpq EndpointSlice for Service rhoai-model-registries/model-catalog: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "model-catalog-b9qpq": StorageError: invalid object, Code: 4, Key: /kubernetes.io/endpointslices/rhoai-model-registries/model-catalog-b9qpq, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 014b344a-dbfe-435a-9325-e70e8fb2e2d8, UID in object meta: 126m Warning FailedMount pod/model-catalog-7cc8755dc9-c9ntl MountVolume.SetUp failed for volume "kube-rbac-proxy-tls" : secret "model-catalog-kube-rbac-proxy" not found 126m Warning FailedMount pod/model-catalog-7cc8755dc9-c9ntl MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 126m Warning Unhealthy pod/model-catalog-7cc8755dc9-c9ntl Readiness probe failed: dial tcp 10.128.2.170:8080: connect: connection refused 125m Warning FailedToUpdateEndpointSlices service/model-catalog Error updating Endpoint Slices for Service rhoai-model-registries/model-catalog: failed to update model-catalog-rlg55 EndpointSlice for Service rhoai-model-registries/model-catalog: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "model-catalog-rlg55": StorageError: invalid object, Code: 4, Key: /kubernetes.io/endpointslices/rhoai-model-registries/model-catalog-rlg55, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 311661a9-050a-4438-8b48-a307f5f54928, UID in object meta: 125m Warning FailedToUpdateEndpointSlices service/model-catalog-postgres Error updating Endpoint Slices for Service rhoai-model-registries/model-catalog-postgres: failed to update model-catalog-postgres-48pzp EndpointSlice for Service rhoai-model-registries/model-catalog-postgres: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "model-catalog-postgres-48pzp": StorageError: invalid object, Code: 4, Key: /kubernetes.io/endpointslices/rhoai-model-registries/model-catalog-postgres-48pzp, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 9c66614a-4571-46a9-8eb2-feafd2196344, UID in object meta: 115m Warning FailedMount pod/model-catalog-7cc8755dc9-blkbw MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 115m Warning Unhealthy pod/model-catalog-7cc8755dc9-blkbw Readiness probe failed: dial tcp 10.131.1.16:8080: connect: connection refused 93m Warning FailedMount pod/model-catalog-7cc8755dc9-wdzcm MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "model-catalog-kube-rbac-proxy-config" not found 93m Warning Unhealthy pod/model-catalog-7cc8755dc9-wdzcm Readiness probe failed: dial tcp 10.129.2.151:8080: connect: connection refused 78m Warning Unhealthy pod/mysql-db-registry-1783406077384-594fc44d89-88vh4 Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure.... 77m Warning FailedMount pod/e2e-test-registry-57cb7c6bb9-lk6vc MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "e2e-test-registry-kube-rbac-proxy-config" not found 76m Warning FailedMount pod/postgres-registry-1783406077384-987f85b98-796nv MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "postgres-registry-1783406077384-kube-rbac-proxy-config" not found 75m Warning Unhealthy pod/default-db-registry-1783406077384-849897cddd-zbb4p Startup probe failed: HTTP probe failed with statuscode: 503 75m Warning Unhealthy pod/default-db-registry-1783406077384-849897cddd-zbb4p Startup probe failed: Get "http://10.129.2.168:8080/readyz/isDirty": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 75m Warning Unhealthy pod/default-db-registry-1783406077384-postgres-7f4d674d67-8wbx7 Readiness probe errored: rpc error: code = Unknown desc = command error: cannot register an exec PID: container is stopping, stdout: , stderr: , exit code -1 74m Warning Unhealthy pod/model-registry-db-662874-7956bc68d9-d4c8l Readiness probe failed: mysql: [Warning] Using a password on the command line interface can be insecure.... 74m Warning FailedMount pod/test-model-registry-662874-748fdd9884-fx2hz MountVolume.SetUp failed for volume "kube-rbac-proxy-config" : configmap "test-model-registry-662874-kube-rbac-proxy-config" not found --- namespace/rhods-notebooks --- LAST SEEN TYPE REASON OBJECT MESSAGE 82m Warning FailedMount pod/jupyter-nb-ldap-2dadmin1-0 MountVolume.SetUp failed for volume "kube-rbac-proxy-tls-certificates" : secret "jupyter-nb-ldap-2dadmin1-kube-rbac-proxy-tls" not found 82m Warning FailedMount notebook/jupyter-nb-ldap-2dadmin1 Reissued from pod/jupyter-nb-ldap-2dadmin1-0: MountVolume.SetUp failed for volume "kube-rbac-proxy-tls-certificates" : secret "jupyter-nb-ldap-2dadmin1-kube-rbac-proxy-tls" not found 74m Warning Unhealthy pod/jupyter-nb-ldap-2dadmin1-0 Readiness probe failed: Get "https://10.128.2.185:8444/healthz": dial tcp 10.128.2.185:8444: connect: connection refused 74m Warning Unhealthy pod/jupyter-nb-ldap-2dadmin1-0 Readiness probe failed: Get "http://10.128.2.185:8888/notebook/rhods-notebooks/jupyter-nb-ldap-2dadmin1/api": dial tcp 10.128.2.185:8888: connect: connection refused 74m Warning Unhealthy notebook/jupyter-nb-ldap-2dadmin1 Reissued from pod/jupyter-nb-ldap-2dadmin1-0: Readiness probe failed: Get "https://10.128.2.185:8444/healthz": dial tcp 10.128.2.185:8444: connect: connection refused 74m Warning Unhealthy notebook/jupyter-nb-ldap-2dadmin1 Reissued from pod/jupyter-nb-ldap-2dadmin1-0: Readiness probe failed: Get "http://10.128.2.185:8888/notebook/rhods-notebooks/jupyter-nb-ldap-2dadmin1/api": dial tcp 10.128.2.185:8888: connect: connection refused --------------------------------------------------------------------------------------------------- ==================== OPERATOR LOG HIGHLIGHTS ==================== ========== ODH/RHOAI LOG HIGHLIGHTS (errors, webhooks, reconcile failures) ========== Source: rhods-operator pods in redhat-ods-operator (since-time=2026-07-07T04:44:15Z) --- redhat-ods-operator/rhods-operator-6f6bb89fdf-59gvk --- {"level":"error","ts":"2026-07-07T06:40:43Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:41:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:42:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:43:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:44:04Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:44:55Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:45:25Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:46:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:46:32Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:47:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:47:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:48:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:49:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:50:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:50:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:51:33Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:52:17Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:52:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:53:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:54:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:54:59Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:55:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:56:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:57:19Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:58:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:58:44Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:59:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:00:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:00:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:01:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:02:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:03:26Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:04:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:04:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:05:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:06:29Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:07:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:07:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:08:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:09:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:10:26Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:10:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:11:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:12:32Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:13:16Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:14:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:15:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:15:46Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:16:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:17:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:17:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:18:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:19:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:19:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:20:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:21:22Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:22:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:22:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:23:43Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:24:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:25:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:25:55Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:26:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:27:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:28:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:29:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:29:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:30:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:31:39Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:32:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:32:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:33:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:34:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:35:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:36:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:37:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:38:00Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:38:58Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:39:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:40:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:41:25Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:42:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:42:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:43:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:44:44Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:45:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:46:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:46:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:47:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:48:17Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:49:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:50:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:51:00Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:51:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:52:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:53:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} --------------------------------------------------------------------------------------------------- ==================== OPERATOR POD LOG: rhods-operator-6f6bb89fdf-59gvk.log ==================== {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:40:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a41bf627-700c-4466-b813-0bbe1d9d96b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:40:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5da56a46-09c0-4c30-9ef0-51c828e9b9d0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:40:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f5ee9b7f-7ea2-4d55-a232-4ff1406a05f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"153e07aa-a879-4fc1-88a5-746bd27ae752","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2cf55d5e-9d6b-46eb-9642-1e9fc6eae102","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2cf55d5e-9d6b-46eb-9642-1e9fc6eae102","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f6ea1ea0-4adb-40c0-9343-af23cda715b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"error","ts":"2026-07-07T06:40:43Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c103c469-d14d-46de-9b96-ed03e1b7715e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:40:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:40:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2db20b31-bb02-4bcf-ba11-9eed0b9158e7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:03Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-llmd-deployment-maas-124811","reconcileID":"23f31d43-1c89-437e-971e-99a5ca3a244c"} {"level":"info","ts":"2026-07-07T06:41:03Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-llmd-deployment-maas-124811","reconcileID":"c9ee4e39-8b68-4f2c-ac87-098904c096a0"} {"level":"info","ts":"2026-07-07T06:41:03Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-llmd-deployment-maas-124811","reconcileID":"ad4257ca-eb54-4a8d-968c-3b9517ab34ce"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1adefde4-219b-44e0-b443-1cd69ea30550","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"df68d563-3ce5-40ad-a8ab-8a622613925d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"lifecycle-test-141919","reconcileID":"52ae973d-4ee4-462c-8614-29790a620569"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f1315f2c-a8c6-4a49-ae25-66201f1ab4d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"lifecycle-test-141919","reconcileID":"a278af86-6fcc-4b34-9731-58336629773d"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"lifecycle-test-141919","reconcileID":"fb8c508d-a362-4656-ad22-85d1d01ae7cc"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"95d44369-a3f3-4818-9964-f48bcc896384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f141a73e-02fb-487e-8179-23ffb1af6d1b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"886531ad-2d51-473b-a196-d6a377f33123","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"886531ad-2d51-473b-a196-d6a377f33123","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb9d7a5a-a53f-411b-9a78-ad56c355be3a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"fe737685-82f3-4614-b2f1-63a26f8194d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:41:14Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:41:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"0063b6fa-4e83-421d-aa1f-9b7b344ce898","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:41:15Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"31f788f3-2d2b-4ad6-b7a5-4b44a04cd6bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:41:15Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:41:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"a90ab0de-275f-4b51-995f-5b2750296877","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:41:16Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"04688bf9-399c-4a12-abc9-0fe48b5bcd69","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"51310b33-a5cb-4f7f-ae5c-e8dc741abd34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:41:34Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f00d669a-0a8a-451d-a204-5d3fbd00f5df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:41:40Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-llmd-deployment-maas-124811","reconcileID":"766a1897-4591-4c01-8af4-31aa68a6f2dc"} {"level":"info","ts":"2026-07-07T06:41:40Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-llmd-deployment-maas-124811","reconcileID":"8abb97f6-eb88-476f-87ac-3d253ee76d41"} {"level":"info","ts":"2026-07-07T06:41:40Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-llmd-deployment-maas-124811","reconcileID":"5859c8e4-c6de-463c-af3c-3d044049785a"} {"level":"info","ts":"2026-07-07T06:41:51Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:41:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:41:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d74b0134-962e-4815-8c2e-6dcf439fa3a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"328d09f0-15e7-4b87-8720-b2f8f36bce29","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9cbea465-c725-47fe-9247-9e251b87f150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"36fe386d-7e60-4c16-9318-718cdb25c040","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:18Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6d3fee15-c288-4e3d-a0ec-2d33412c7602","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dc0b7232-4e2b-453d-ab20-c92d2537e92b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:18Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"38198a08-1ba8-4a87-8445-b63ed6d2194d","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:42:18Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"38198a08-1ba8-4a87-8445-b63ed6d2194d","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8e0d52ad-eaef-4c7c-8085-04377ad318fd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:19Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4c8897f6-3684-4624-9a38-c2adbcea3faf","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:42:19Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4c8897f6-3684-4624-9a38-c2adbcea3faf","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:42:21Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"hw-model-deployment-116404","reconcileID":"f8bc7e06-f25e-41de-9602-afc57511b7f4"} {"level":"info","ts":"2026-07-07T06:42:21Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"hw-model-deployment-116404","reconcileID":"ecb403bf-96d0-48a1-a108-60b5d2f2fabb"} {"level":"info","ts":"2026-07-07T06:42:21Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"hw-model-deployment-116404","reconcileID":"8e8e77c4-cab9-4d53-8ba6-194b6e9b9a0c"} {"level":"error","ts":"2026-07-07T06:42:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"845c27b9-461b-4c6e-a070-42943fe8b77d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:30Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"57c5b3d7-4cea-415a-9b64-6b9ccb776dc2"} {"level":"info","ts":"2026-07-07T06:42:30Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"586f2b49-0ba9-47c9-b778-754e8a728bf1"} {"level":"info","ts":"2026-07-07T06:42:30Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"e14a5791-1e74-4fb8-804b-ef3602bcbed2"} {"level":"info","ts":"2026-07-07T06:42:34Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"hw-model-deployment-116404","reconcileID":"8a783798-ddb7-447b-a33b-1ebb1158fae6"} {"level":"info","ts":"2026-07-07T06:42:34Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"hw-model-deployment-116404","reconcileID":"9b30e8ef-26bc-447b-9040-1b91a5afbb0b"} {"level":"info","ts":"2026-07-07T06:42:34Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"hw-model-deployment-116404","reconcileID":"6691dd87-6c5c-49b6-a24d-fb7be4ea396c"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"285c15cc-fc6e-40b4-9389-f1a595e1571e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:42:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:42:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"40a879ab-d896-4df1-a456-83fa8eff686d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:43:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"726b04ef-b439-4ed3-bf71-77db2f03e2b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:43:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:43:22Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"trainer-v2-test-113378","reconcileID":"db0a6782-da1a-4b3c-8327-bb5e4ce50511"} {"level":"info","ts":"2026-07-07T06:43:22Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"trainer-v2-test-113378","reconcileID":"bf9842f5-a558-4b87-a98d-190f0a8825d1"} {"level":"info","ts":"2026-07-07T06:43:22Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"trainer-v2-test-113378","reconcileID":"cd6799e0-9e36-4586-a3f8-80888e2819e9"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:43:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"578b9df6-22e4-430c-bb34-679a402365c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:23Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:43:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"027b4477-dad5-4f06-9138-73132cb0973f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:43:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:43:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e3db3dc0-bf02-416b-9e60-64bf689f32bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:25Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:43:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:43:25Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"66279d39-c950-4be4-83ce-2e3cb73914a1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:43:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bffd7e92-eec2-4d13-989e-dbd2d238f07a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:26Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:43:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"7d609e53-7ec6-4d2b-8889-2b0fed38027c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:43:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:27Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7b2cf920-b22a-4fb9-98f4-bab66ca7163a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:43:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c362f9c-c26f-4240-a14a-8f132f64a326","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:43:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:43:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"54dc1543-41df-4e53-863c-add9d9ced502","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:44:04Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:44:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"649d8ec3-c904-4b45-b35b-3959972011cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:44:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3b3a1434-ba55-485f-8a0c-d401b00c7d50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:44:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:44:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa75a7e2-d150-4025-8b3e-446f77451ca0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:44:55Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:45:25Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:45:28Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cypress-test-delete-applied-hp-105453","reconcileID":"e08f3926-a79b-4e61-977d-b05450f6741c"} {"level":"info","ts":"2026-07-07T06:45:28Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cypress-test-delete-applied-hp-105453","reconcileID":"93ff738e-7bf5-450f-9e59-f62af86459f8"} {"level":"info","ts":"2026-07-07T06:45:28Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cypress-test-delete-applied-hp-105453","reconcileID":"370d6a20-6f19-46b2-a193-12dff483beca"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:45:30Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:45:30Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"da42ebc9-c750-4914-bf69-a0ba12e034f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:45:31Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4f3c0532-e639-48b5-897e-e76dd9df404d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:45:31Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:45:31Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cff2c9d6-f257-4a88-bf14-cc39a5679585","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:45:32Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0d95b0d8-b327-4b93-b682-adc88d4d7f39","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T06:45:32Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T06:45:32Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"d87bef76-6d55-45f1-bacc-4103327346c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:45:33Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:33Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f56ad787-ff6a-4f79-a372-1eded77be12f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:45:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71a00fbf-5710-499a-b168-9ecf1a8a80f5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:45:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4e0f7df8-8c18-4131-a2b4-3c9c462d054b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:45:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5fde75e1-e9b6-4f42-b027-20afb3719fe5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"32402daa-5eb9-4a8b-83c8-ee28781d1f20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ee8b699c-a082-4cf4-bf53-8da1d87d5672","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ee8b699c-a082-4cf4-bf53-8da1d87d5672","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:37Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a1108351-a8df-4ac4-9eec-14e20eb6eae8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"78728825-cc74-44fe-b057-a0e89173caab","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9b2d4d24-a863-4252-8a7e-9c9d82eafe33","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9b2d4d24-a863-4252-8a7e-9c9d82eafe33","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4f4c3294-f3bc-4ecc-b41a-cfdbabc15e68","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:52Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:45:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:45:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b763e40f-731e-4716-a129-44e3b50d1de5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:45:56Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T06:46:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ca92de11-0fc1-419d-ae22-577da09a83ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"29c18da7-b843-405b-9550-b8738d436217","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e100ded-5dce-456b-836f-7f2104fc06b4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"14fff525-ea09-4ed4-8a67-af964766d7f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"de990334-1889-4beb-801a-5a9aaf8ca2ec","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9b48b428-ac8d-46bb-b448-82bb3a447a19","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9b48b428-ac8d-46bb-b448-82bb3a447a19","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"34a38bbd-d777-4b15-9349-c66ebcba10b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7840348b-07da-438c-acaa-6def5fd7118f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ff37f735-c836-4fc6-be08-b333ac30a06c","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ff37f735-c836-4fc6-be08-b333ac30a06c","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21999ce4-e201-4638-8767-492d284393f2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"8cfbd085-15a8-43b8-ab8f-c67c52493ffa"} {"level":"info","ts":"2026-07-07T06:46:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"1893fedb-839b-4086-b9ad-3d19ed11123d"} {"level":"info","ts":"2026-07-07T06:46:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"6577cbf5-e71b-4aa9-8b88-7d5249169ff6"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3c634e43-7ded-4a51-aa60-bc67a944bbfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:32Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cfc5cf13-c86c-4a83-8fb9-c43b4c4eab96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:32Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ffee89ab-f324-41fd-8f7e-801640e866f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:32Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8f81db9e-d398-4cf0-976f-73d5c65f93ac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:32Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5bb21a3-29ce-4aad-a2b4-3131ee1695f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4c41b430-fbf4-41ce-9a88-4b5c34e93c9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:32Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"02873bd9-9d5e-47b2-8f75-11a9c72a1c92","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:46:32Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"02873bd9-9d5e-47b2-8f75-11a9c72a1c92","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-07T06:46:32Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"84fb00da-bc37-4ad4-824d-9bcdf79ee5dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:46:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:46:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e20ac1ff-3399-4976-a639-ff46b53cb2fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:47:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:08Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:08Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"89b80ba2-1e73-4206-a316-640ddf30003d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9b84604f-5bb7-4cc9-a4ee-b010424c672b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8215db60-638f-4531-a6c5-1230dae8e6a2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19d28c86-4565-4a02-af9d-5abebaa76bba","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:25Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"62b40727-802b-4cd8-bb9c-e1b2d78289ec","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:47:25Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"62b40727-802b-4cd8-bb9c-e1b2d78289ec","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7bbbce6a-1beb-4835-89d1-503126da7ef5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2cebcbd2-5779-49fb-8297-4c3cac2c8446","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f8c91215-074b-4c77-803e-bca4c2b9313f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed865f83-1998-4b9f-8a64-5fb2d847e616","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:28Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"789c30cc-b919-4319-bd0b-6b08c4050af2","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:47:28Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"789c30cc-b919-4319-bd0b-6b08c4050af2","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:47:36Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"cf8ba5b9-c1db-40b9-81c7-0b788f5a39e7"} {"level":"info","ts":"2026-07-07T06:47:36Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"d1943cdc-bc91-4e35-a4ff-01a3bf7fd456"} {"level":"info","ts":"2026-07-07T06:47:36Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"dsp-hp-modify-test-159216","reconcileID":"46f145fb-f074-4af4-a3bf-f7b1577b252c"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fdf547ad-e4aa-4d58-8dd2-513506ba8481","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:47:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9be8b1fc-a964-4a63-8f3d-abf22c7fdd64","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f65801c0-6f2e-4780-a969-ea4140faf576","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:47:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d347e97-ebf0-4d6d-aebe-2cd73e3b8a96","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"0c7aa033-e4ba-4c66-b991-ea1fc0af7b01","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:47:52Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dfe2665c-326a-42bf-9ed5-1a2d8bc0951c","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:47:52Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dfe2665c-326a-42bf-9ed5-1a2d8bc0951c","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"18cb535c-d4fb-41af-bdf4-360e60a02106","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f2026029-0135-4285-8b6f-124814ccbcee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:48:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e135dd89-f1e9-4580-aa49-6dff89710bbd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d682817b-47a8-4e4d-907a-568ed65cadfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"630427c5-0594-4629-8341-3fa75b4cc7f2","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"630427c5-0594-4629-8341-3fa75b4cc7f2","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7f073f33-d9c5-4ce2-aef7-d1870b2545f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cf3a64a-50af-4592-b108-27d02646c58d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"884bf2e2-c3cd-4418-8ebb-ded8d9cc31f6","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"884bf2e2-c3cd-4418-8ebb-ded8d9cc31f6","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c8b7fb2-1c04-4fe0-b0d7-8c96886eec7e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"74ffb51f-df7e-4844-bd1b-26cb39f96d5f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a5c34ab7-ecfa-4c2e-9be4-7b26ff7923bf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81ca51a7-7436-4c1e-bd52-0ea232d6aa81","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71b4dc4d-122e-4477-870c-caa545817663","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c7e83288-e172-4eda-b191-785db6aac1f1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:48:40Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1d637e35-fc1d-4f64-8fb5-43539f1fd689","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:48:40Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1d637e35-fc1d-4f64-8fb5-43539f1fd689","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:48:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:48:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c9922655-63f3-40a5-9af0-68756a79cc67","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:49:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:49:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"03f7a710-21be-4410-8027-7f3338bec53e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:49:24Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:49:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:49:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"53a82ea8-4f56-4fd3-86f3-8c68a29371b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:50:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:50:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2d19e6a5-b881-4708-90c8-ff362da9888b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:50:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b865e836-9c73-463c-af4f-23fada5226c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:50:27Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:50:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:50:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b4115c57-bc3a-4f5f-a2e8-b9f3b1b8a219","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:50:41Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"1c05d312-40aa-4479-b31c-60d593551ede"} {"level":"info","ts":"2026-07-07T06:50:41Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"47670a84-6b2b-46f8-a821-a3d236aff7c2"} {"level":"info","ts":"2026-07-07T06:50:41Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-custom-pip-prj-150747","reconcileID":"950a4406-d6c1-4b67-a2de-d5495f921526"} {"level":"error","ts":"2026-07-07T06:50:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:51:03Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:51:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab6c7f93-1677-4c6d-a2f1-c60255b8a082","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:51:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6361415d-79f7-4854-91e2-80e49171d52d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:51:33Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:51:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:51:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"adeadde3-e6a5-4026-8b7d-14e9331d026b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:52:17Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T06:52:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:52:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:52:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7e2b58ff-1f64-4876-a736-a5bbc3f99cb1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:53:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:53:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d12712cc-e9b7-4a8e-b99b-29c99cb1428c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:53:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:53:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:53:54Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"faaaa5c1-1ad2-425b-9537-ef16a8042daa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:54:07Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:54:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:54:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"758111d1-b497-441e-a191-e6b2faa4855f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:54:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:54:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9aaf5b92-c604-4ea1-80f0-53d52e98afdb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:54:55Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T06:54:59Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:55:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e85c1585-9dcc-4fd9-9e15-764cde71021f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:55:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4c9ac130-71fa-4a70-8080-18b8c15eac99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:55:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:55:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:55:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"693d49e8-4742-48a5-beef-1fd8259891db","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"8c0ba906-95f8-49cd-86bc-fa9008454150","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T06:56:00Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:00Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0c857f85-5142-4d93-a920-ef7e0cd6aa72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:56:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c673c228-e2a7-4fa3-8762-0f225f7d93dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:56:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:24Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9ab61d56-1123-4b27-a43d-10b940fa3074","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:56:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:56:39Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2"} {"level":"info","ts":"2026-07-07T06:56:39Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2"} {"level":"info","ts":"2026-07-07T06:56:39Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:56:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0de4d04b-0500-48da-8525-5065616abce2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:56:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:56:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"55211104-78d2-4e91-958b-2e0bd2b98cd7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:57:10Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T06:57:19Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:57:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:57:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cc72599d-f219-4a96-980d-c362a419557d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:58:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:58:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:58:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dcc42b33-57dc-474d-a4c4-f6e090664048","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:58:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:58:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e00ab8d-d3f9-4035-8f10-c62232d8ee72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:58:44Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:59:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0c1c13fd-906f-4b6c-886b-b849db7aca9a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T06:59:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"015b173a-d1dc-4ca1-938e-b02bec2efa59","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"65d50c8e-ab90-4125-8482-90af3a1156c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"19f57656-52d9-4f03-bd37-d1dda7e9afb9","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"19f57656-52d9-4f03-bd37-d1dda7e9afb9","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T06:59:10Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5864b7f9-78b6-4899-9086-51543b9c0528","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T06:59:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T06:59:35Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"169cc34e-ec77-46e2-819b-fb67d2c59d2f"} {"level":"info","ts":"2026-07-07T06:59:35Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"f00c1cb9-412f-46c4-afd6-68d6193936ca"} {"level":"info","ts":"2026-07-07T06:59:35Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"1394228c-66dc-46a7-b6d8-d33f7b7db9e8"} {"level":"error","ts":"2026-07-07T07:00:03Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:00:04Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:00:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"43ae2571-2f01-4bfe-b47b-7a9534aaa3a8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:00:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8fce2d7f-4020-42bb-835a-637b5473e7b8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:00:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8ad4163a-58e9-4d71-974f-746a0fb47c52","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:00:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:00:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ed2a284d-30fa-4602-87eb-ed25ada5a548","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:00:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:01:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c90da594-caeb-4519-8329-cc3ddcf48a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:01:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2b1b0865-eb83-415e-a729-f040783f9f78","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:01:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:01:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6f71fcc4-9362-47f6-a834-3363dcf7df37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:01:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:01:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bac4328-bb16-4cb0-a45b-ee3343b01461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:01:58Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:02:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2c4d7ec5-2bf5-4341-8b6e-b61e9ee762ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:02:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d84a5369-b3f3-4346-819b-208ffb18797f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:02:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:02:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c4321ae2-3e05-4cf6-ae37-b247195df509","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:02:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:02:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"dfecc338-d939-41b3-9778-d7a067cf29c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:03:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0529006d-3bd5-43bd-8c41-0f2bee14fe5c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:03:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66574035-6f79-449c-a2f0-9f5e9682bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:03:26Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:03:26Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:03:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:03:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d1cc6aa8-0fb3-49b4-b28f-2127193283cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:04:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:04:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"155d7170-f8a8-4051-9a0e-e3b84667ef43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:04:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:04:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5922aabc-b4f4-4e15-b7b6-9cc0442fbcb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:04:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:04:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fce75d92-b2d0-4052-ad86-408093e0946a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:05:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:05:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8db4c53a-c97b-4fa9-be2a-3b2ff402e053","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:05:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:05:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:05:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48c962fb-38f9-407a-8896-a8dd87fc3532","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:06:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5b62ab88-424c-4fd0-ba58-8defbd7affcb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:06:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8e9796bd-c933-4f1f-8b9d-4d1bdf2ae698","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:06:29Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:06:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:40Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"99801b5a-7ad8-43a8-a0a6-62397e7fcdc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:06:55Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:06:55Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cef8c253-1005-4baf-b374-1c484bf0691a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:07:10Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789"} {"level":"info","ts":"2026-07-07T07:07:10Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789"} {"level":"info","ts":"2026-07-07T07:07:10Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:07:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6ed440b7-fb0a-4c06-9d0b-0c24fe3a6789","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:07:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:07:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"761eafea-591e-4065-8e37-de1e61a03723","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:07:39Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:07:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:07:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e40a70ca-2de8-472d-aa76-a196ffabcdbb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:07:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:07:50Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"1d41a616-0be6-4329-b441-b3f86d64df83"} {"level":"info","ts":"2026-07-07T07:07:51Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"e9d915b2-b106-40df-8c39-882e25671221"} {"level":"info","ts":"2026-07-07T07:07:51Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-pipelines-prj-643148","reconcileID":"04f50eac-7103-4165-991e-a1ea45bfd9c8"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:08:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea14d8a1-6ca8-47e7-b681-c208bba686b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:08:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b38c1536-b1fe-4591-9b32-ad789aba245e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:08:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"52e982ba-d0ba-4e82-9eb4-8b34f8e151d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:08:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:08:56Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:08:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c36bf1a0-5398-47e7-af68-c3a3912fe887","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:09:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:09:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"789a11ce-cdda-4bb2-a6f0-adf8d802f2bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:09:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"2fa1c2b8-43e8-41c5-8aeb-8ac6267c8a95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4fab0c0d-f55d-4b7d-b376-2d54cbe6d5e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4cc98e0c-d90b-4d34-bb5e-1cf5ca146759","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4cc98e0c-d90b-4d34-bb5e-1cf5ca146759","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9a85e0a0-d493-43d1-b257-e2e392c89229","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54fae083-f0a2-4184-b16a-d6e39033ee50","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e075d13c-6c09-422b-83c9-cbfe6af67b04","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e075d13c-6c09-422b-83c9-cbfe6af67b04","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"64bb9d22-9011-4ba1-9b89-34fa140c4842","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:42Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bd80a6e5-e738-4214-bfa2-d4b002f7542e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1601f5a1-fa36-4afd-aaaf-c055cba5a61e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2acbcf82-fd10-49ed-b58f-d74fd8e55de6","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2acbcf82-fd10-49ed-b58f-d74fd8e55de6","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:43Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5e845659-a0fc-48af-abef-f85fbd23dba4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:09:56Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:09:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0910a0f1-f34f-4667-b0b9-a0a7e8baa5a9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:10:01Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:10:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"34b03dd5-9e62-4b4d-95a1-36145995e600","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:10:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a77b6f63-8037-4265-9861-198990605f12","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:10:26Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:10:56Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:10:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b1e43945-670e-474a-924a-09705d6d9c16","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:10:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:11:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:11:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:11:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"0eda3195-f359-4127-a465-3e869de44e0c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:11:37Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:11:48Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:12:32Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:12:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:12:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"17b8609f-eae9-461e-b3d8-f6b8d3cdd0e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:12:56Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:12:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"198c60e8-65bb-491c-892f-987cb3242afd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:13:16Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:13:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:13:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa1756db-7255-4d97-a14a-93965f859438","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:13:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:13:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"aa485777-8482-4ddc-af8f-63e5a245070b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:14:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"49579a39-3b3a-42c1-a233-8ba24caf88a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:14:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:14:26Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:26Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6bd9c283-5cad-4e7e-abfd-d36b43afe8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:14:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:14:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"040643c3-3692-4da5-9413-e5bd9a194ab5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:15:01Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:15:41Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:15:41Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7cf41256-8f4e-4285-9047-c3c842590ffc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:15:46Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:15:56Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:15:56Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"cdaf0317-82e9-4b43-923f-8adc6e298c73","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:16:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:16:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"8b5964b6-2114-448e-900d-1a4d7f20d36a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:16:14Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:16:16Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:16:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:16:37Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"ea5c7c92-bdaf-4c5b-aa0f-9536eb02c80d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:16:41Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"fb28a8d8-ee33-42e5-92e9-0151cf467336"} {"level":"info","ts":"2026-07-07T07:16:41Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"70ede39d-0a66-48c2-9b7d-b3ddec4bf839"} {"level":"info","ts":"2026-07-07T07:16:41Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"e003ef14-66d0-46d4-8bdd-0e5cc4a6f9b4"} {"level":"error","ts":"2026-07-07T07:17:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:17:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"81370338-dee9-4acc-b450-056d9c0e2594","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:17:29Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:17:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:17:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f25814ab-ad34-4270-abd5-9aa39c2e7720","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:17:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:17:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a4546bc6-eb1e-452e-8c12-03e2c57313dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:18:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"48822183-2edf-48ac-84c5-2b3414521e27","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:18:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"699060fa-184f-428d-80f7-076891cab1ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:18:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:18:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ad454a63-dae3-413d-826b-2140a1648237","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:46Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:18:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"8c20fd16-bf57-42c0-98b6-b4981b207d3f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:49Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"874889b3-b674-461c-8938-4d0a24c8d59b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"5c5a9e2d-9462-4d9a-a966-0137badd3568","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"36e52408-3650-4fd1-8d81-b4a18eeb6107","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2244343b-2358-4a36-a57b-98cc9a4ec993","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"af89d0cf-d066-4cee-9657-2ec63baa449b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"75948cf3-67ee-47c6-b77f-a818f7a8aa9f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"9c2d2b14-9934-41f9-9a91-8a15511b83cf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"468f9319-088f-4f46-9054-7b9c5212f96a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"f4828a46-0b75-423e-add6-5eeebbb17d4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"f95c1b79-455c-46d0-8fd0-471b69d71861","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"970dcc18-7955-4dd4-9856-da7214d68566","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f9331766-ce66-4ac0-9b86-bebee95f6a61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"92703342-26ba-4db5-8969-5b6caf6ad1e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:51Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:51Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:51Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"547fac91-5afe-4c06-b1de-6e29f48a8461","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"64ba60fa-9a46-45a5-93fc-cadad56c4e72","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"bdebce32-e2d3-4a32-ad41-6e801b4d3c76","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"0a95910b-c9ff-472a-8193-35f1d4af0e9e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"e18c8d21-49aa-4070-acae-dc1ff638f1b6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"9fbac315-a216-4ffc-883c-166e9172292e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4185b670-1446-431e-82e8-98f737576be6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"ed5522af-d6ca-4942-b90b-54aed08cbcb6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"68dad601-5f5a-4924-ab2f-bc6322ae72cc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"47e80299-5875-4e6c-ae1d-5f062b91253e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"c04de7d7-6394-425e-8355-c5bea8af0378","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"3df264b9-69be-49de-b8e7-c818e410bf3d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"1447ea71-8e41-4d30-8e27-b097fb9aa47e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"6dd7afb1-3257-4dae-950b-1cdc428a4761","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7fc9632b-1973-45a6-a4c4-eb069839740f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f7736be6-3ef3-4e9c-8b0e-0c59b6ba9cef","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:18:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"dc0f87ab-ead7-4211-a846-db1da5fe7818","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c8611bc6-6979-47e0-9b22-e5c30468ba0d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"93eca0e3-b3d5-4290-94dd-43909f420926","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"24502abc-dd16-4bce-aee4-d05b1b90c827","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:55Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f0e1f271-311d-4f47-acc5-9ec42a3219fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:56Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:56Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:56Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:56Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-07T07:18:56Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:56Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"b5f881a2-01a9-4363-a864-7c27d830709e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:18:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f199a163-d25c-4a0e-a341-8e4ae5a80d37","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"c559884a-9087-4134-9256-1b545016d541","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:18:58Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"db4b802d-23db-450b-870e-70f61cfc59b3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:19:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:19:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c65b06c5-b39b-4b5f-8331-90dbfa3d81cb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:19:48Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:19:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1124f8c0-4a72-4b8a-8e0d-9db69959fd06","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:19:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6af8ddca-4e22-46dc-aece-8685ca7a2dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"dd009eef-46f0-49eb-a513-9f3362b9999c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4330983d-a194-4479-b59d-2db1fdd96aaa","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"4330983d-a194-4479-b59d-2db1fdd96aaa","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:57Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a02062bb-7e3b-4fb8-8200-b851e0a10b04","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e203a7a3-9cc1-4ad2-b08e-b7cc8c417f95","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ce8f308d-0606-4b35-98d6-47d1659937ad","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"ce8f308d-0606-4b35-98d6-47d1659937ad","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"01bd7f08-f446-4756-86ff-43ccef0c51a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"878142d6-2b4c-4cae-a938-f9dcfc87b271","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"52d975c2-3909-4916-8e58-76571e2a23f6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"db2dd70f-00cc-48e9-8584-e039b4a550bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:19:59Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"625fb054-20ca-4803-9754-84c16853f5a1","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:19:59Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"625fb054-20ca-4803-9754-84c16853f5a1","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:20:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1440a2e5-5d65-42c4-91c5-6b3bbe768f70","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:20:25Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:25Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"689c0de1-48f4-41e6-ae9a-c8847e77ec18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:20:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"729e602a-62ff-4934-86c5-2e9545adeb4f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:20:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:20:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:20:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9640e918-d002-4951-8492-1ecef551c85d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:21:22Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:21:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:21:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:21:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d78dfff-e30e-43ac-89dd-e948cc3d6d7a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:21:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:21:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3010c740-6557-4b99-9630-c5d1e94da82f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:22:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:22:49Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:22:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:22:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ec8b50f3-f5b3-4bf0-9ab9-a77a9baf004c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:23:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:23:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"893cb1d0-161b-431a-afd8-9c4f012db10c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:23:43Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:23:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:23:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"45fa4303-dfc2-44de-89e2-a49613d9997b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:24:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e861c495-8f35-4a24-b31c-5e48a310192d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:24:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ab1188b8-d5c3-408d-af2e-9fcb9a2a668e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:24:41Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:24:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:24:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"d8737877-b523-4eb5-8f16-49c7c69d5794","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:24:57Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"852b04cf-238f-49e6-831d-56259ff30129"} {"level":"info","ts":"2026-07-07T07:24:57Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"5d37b999-83b5-4eb4-bae5-7cb9512cc81b"} {"level":"info","ts":"2026-07-07T07:24:57Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dsp-schedule-prj-115465","reconcileID":"64cfd04e-22c1-4058-bc62-f7b401171de9"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:25:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"3bf02603-75c6-4f0b-b225-f950cd3d4919","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:25:14Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:25:19Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:25:27Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:27Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e2d693bd-748b-4afb-a22e-2fe04d55f9bd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:25:42Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:42Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"bd2d5a55-9eed-4fc8-9a8f-cb880e235377","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:25:55Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:25:57Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:25:57Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"f4dae7d5-1599-4f30-baed-8bf8e510b9d4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:26:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:26:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"70543967-9bdb-4570-9eb7-db1301e76630","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:26:15Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:26:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"9d221b9b-d21a-402f-a3b2-ade546a402c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:26:52Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:26:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:26:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"fd582844-1566-4612-a69f-2f01ce68a743","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:27:17Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:27:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:28:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:28:28Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:28:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b29698af-123d-467c-b752-4ee95f55dc71","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:28:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:28:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5651ba65-778d-44b3-a61a-9e6bee1e476b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:29:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:29:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"68624d09-77c3-4acd-9b1d-3557a4e6b6f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:29:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:29:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e130d8cc-15e7-4271-b143-c0ada7d20037","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:29:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"71d0da6d-9961-4a22-b941-5a46e7db777a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e57212d0-2afe-4405-88a4-6d9f83d5a1e4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:29:58Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"891a656a-eded-42b4-b8c2-f3d7b39a03c9","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:29:58Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"891a656a-eded-42b4-b8c2-f3d7b39a03c9","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:30:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"66f1497b-92ff-4367-a327-5cc11c01a119","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:30:28Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ea9e2fb2-4126-4aed-a208-5bb2d76eec93","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:30:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"93d6167b-63b2-4733-b764-613b6a8ba0c7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:30:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:30:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:30:58Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"4cd1a607-bee1-4ac6-bfd0-73144d1a2039","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:31:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:31:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"a74a4596-901e-47b6-b68c-ce0344d5a092","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:31:39Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:31:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:31:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:31:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"e149ef65-d6ee-4c74-8c06-9bae41738674","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:32:02Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:32:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"c0e7d95e-b543-4ebf-a75f-84bdf6f6fe84","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:32:13Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:32:28Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:28Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"97acd8e6-ae33-456b-a94a-66cda27911de","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:32:43Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:32:43Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"6559d22f-4565-4698-bf16-967024ea47e1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:32:56Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:33:06Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:33:51Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:34:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:34:39Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:35:30Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:36:15Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:37:05Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:38:00Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:38:46Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:38:58Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"62981702-6db8-4c9d-8bdd-132a3dbd861f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:38:59Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"de514d5f-4b0e-462f-a064-5639e9973631","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:38:59Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"de514d5f-4b0e-462f-a064-5639e9973631","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-07T07:39:45Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:40:38Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:40:38Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:41:25Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:42:23Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:42:57Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:43:00Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527"} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527"} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:43:33Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:43:34Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:43:34Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:43:34Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"4813af89-efe1-4e19-aeec-c1c71252d527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:43:54Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:43:56Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2448305e-26ba-4376-bfb6-aacf2ccc989c","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:43:56Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"2448305e-26ba-4376-bfb6-aacf2ccc989c","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:07Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"b19d5ff9-c13c-486b-a30d-8c43c6d3ead2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"b73d9947-3802-4dd6-9a22-03795b7c5e23","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:44:07Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:07Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:08Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0b7c5126-2921-4634-91a6-dbb79daee910","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0b7c5126-2921-4634-91a6-dbb79daee910","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-07T07:44:08Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"972b4e9e-11e0-46a7-a551-c7367f36eddc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:44:09Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"93bfd63f-deea-4ae3-b2cb-ba67bffc15dc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:10Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm","msg":"EnvoyFilter exposes internal implementation details that may change at any time. Prefer other APIs if possible, and exercise extreme caution, especially around upgrades.","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b"} {"level":"info","ts":"2026-07-07T07:44:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm","msg":"delete","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","gvk":"components.platform.opendatahub.io/v1alpha1, Kind=SparkOperator","ns":"","name":"default-sparkoperator"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"da26decb-5c7d-4e3a-bae3-df4e796b769e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b5ee8abb-0564-4198-8cee-3235f7caca95"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6d26ed99-c614-421e-80a9-a335aae1ebc9"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b91a2d20-881e-401a-8a84-ef12e3c6881e"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"b4ba45e8-e247-4148-88da-6e78f3e62c1b"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7c5110c6-1e92-4bd1-9f5c-9fd3526fd561"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"3e4d6d99-4806-49b6-b1df-d0afac7b017a"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46a1173d-0ec6-4ca5-bbc2-4c08c1df8ec7"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"212d83a2-d183-4c2c-b970-95a46354dc0f"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"36333bde-5aeb-41d8-936f-a08975af38f5"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"4ca9122c-fc34-4048-a10f-4661ff7cc15b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"04d5c676-327c-41e2-8d00-511a2d63e1ee"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"77d8ebca-bf45-4f86-ad98-1db03437f20e"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2ac47789-2b6f-468c-aedf-6e6c94fc16e3"} {"level":"info","ts":"2026-07-07T07:44:11Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"007bf79b-f613-4211-95bf-4911a0791892","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"007bf79b-f613-4211-95bf-4911a0791892","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"87e6566e-7e28-4e78-8abf-dd2fd5ae1c0b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"680f6918-9a75-46cc-adfb-71dcec1221e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6f5bf898-be5b-4afa-8e8e-8fef2141ec8a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:11Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"be74ab0c-68ef-478f-896d-8855f5fc82ad","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:44:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"1c862cac-e571-449a-9760-df10c4c197c9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e168bcb2-9526-4fc6-a9ed-1afbb81d5993"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"30f6ef9f-ee02-4b97-b830-893932a58317","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"7062109e-810d-4a50-8df8-20f760731560","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:44:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"d11b5b46-8092-4400-9a94-d801ceb1fd8d","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"d11b5b46-8092-4400-9a94-d801ceb1fd8d","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e2ff8847-e0a5-4abb-893b-b9c040038099","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"ccc9d193-46ec-4b68-b636-41295ad3085d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6aa9b69f-b4b7-4187-b4a1-897017ff376d"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"3a75a3d5-43f0-4208-9b90-cbfa45f30679","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b7361153-5bd5-4d48-8407-0eba54d43533","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b7361153-5bd5-4d48-8407-0eba54d43533","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"bdbacdc0-cc3b-4bf1-9af9-fd771e2926a7"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:12Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"cb4661a3-79a3-4daf-af25-661cbe237a99","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"8f9f3a7d-d10e-4709-b960-887c393ab87e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"6524304f-94f2-48d2-aa69-f76f6ba06374","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"feb072f1-cef0-4148-b5c4-81455a1f3c43","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"670fb761-c293-4ba6-af0d-da4b56f43544","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"30a1ce53-a234-47ae-bea6-9e0774b37b63","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"30a1ce53-a234-47ae-bea6-9e0774b37b63","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"38c6c3f2-5f63-4142-bb9e-8c3287081dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:44:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a8f8433d-9566-4566-b699-0470c07faf56","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"54f835c5-ba1e-4ab6-9644-7662e2f9d34e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"3f80b162-d6bd-4f87-a165-f531c7864382","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"7fe3df3d-346d-4dc0-adce-7d734617eeee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2bb765d9-ab3d-4a8b-be60-001c4a5565b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e910cf89-ec15-48b8-b8e2-a1ab8b69364b","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"e910cf89-ec15-48b8-b8e2-a1ab8b69364b","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"eb2a597e-a064-4bc0-8bcc-4934dbb5405a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:44:14Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5f4a0dc3-236d-4959-b182-727c56a3fe7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"46469d81-03b9-4e70-8e7e-9670937c516f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:15Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"f34c12cd-c78b-4dd9-b844-d51127379e5a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"fce0c21f-4617-449c-a99e-bdb87b5bf826","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:44:15Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:44:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"ec2723ad-7ae8-4a84-a573-593130fd1194","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:16Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:16Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"c6c9c62d-f93f-48ba-9f31-2b2742fb2879","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:17Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eb2f18a4-892c-441d-858d-9c8c84bd5a58","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"12ebc9e8-0266-4251-8f26-df021daeb155","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:44:17Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"15d6cda6-f1df-41d5-bc31-e73f19c91bc4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:17Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"b0f3e4df-92a3-45ad-acf5-7458c6a6d54b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"93c05239-d6bd-4698-88bf-6abf9cf40546"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"93c05239-d6bd-4698-88bf-6abf9cf40546"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"93c05239-d6bd-4698-88bf-6abf9cf40546","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"3fd6cb38-0342-4940-8f43-9416b4192352","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"0ee2451c-6094-463f-914b-cd4b8641f685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9b22b8da-570f-42dc-ad18-6b9c139c643b","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9b22b8da-570f-42dc-ad18-6b9c139c643b","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722345af-52e1-41ee-9236-ead92641ca5b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ce082974-24af-4f26-a865-8531a35b65b0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"7d5bee1e-3c16-46cf-ad7f-1074f4978966","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"cde95503-3c12-40d2-bd73-e4dd2751c4c3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:44:18Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"d1c828f8-145f-4227-ba5c-9ee407bc951e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"b42189b6-8e02-449d-b773-5fbf16305657","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"fc44179f-042e-4272-aab9-836b7022d1c8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"93c05239-d6bd-4698-88bf-6abf9cf40546","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"93c05239-d6bd-4698-88bf-6abf9cf40546","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"93c05239-d6bd-4698-88bf-6abf9cf40546","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:19Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:20Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:20Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:20Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:20Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"d74bf065-a32f-46f7-9c76-0e49da2fdac6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:20Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dd82a2f8-6334-4666-b092-cea5987d76ea","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:20Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dd82a2f8-6334-4666-b092-cea5987d76ea","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:20Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"26b76f76-d8c4-4738-bfcf-8293aa4ca0e8","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:21Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:21Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed4431a7-5736-478c-9dd2-b7b5debf4d20","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6a56938a-eff2-4535-9849-32a047694e98","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"6a56938a-eff2-4535-9849-32a047694e98","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"fbb6bbf9-6a92-49b9-9d09-9dd9b7014acd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:44:22Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:44:22Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"48d2b61c-e03a-4738-a3f7-e0fac0c4ff1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"eaa75f89-d3c6-4fc9-bfe7-f04fb81e5a6c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"23966f2b-c417-47ee-ae88-ea526caaefe5","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"23966f2b-c417-47ee-ae88-ea526caaefe5","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"875ed146-cf53-44cc-8335-38d0baffb570","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:44:23Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"962a7087-6361-421f-b4cf-4bd63c28c71d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:23Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"d7e2286d-15bc-4de6-85ed-97b2d2285fed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"cf6007ee-bc28-4bec-a48e-6a7b501b20d3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"be515851-9882-41ae-bd9d-1f6d26ebcf36","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc02ac1e-1b33-46fb-a090-30b7bfd53d77","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"13e65e8e-49f1-408f-831a-49c7db229049","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"aac1727a-a604-4861-b95c-649bfd982796","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"9b6197ba-fcfe-435b-b193-911cdc27796d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:24Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:24Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:25Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a46723d-f5fe-43fb-a9b9-f2e4f3c978e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7ce55ea6-7124-4818-9061-be409e82e7c1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:35Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:35Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"bc2985f9-0219-4ef6-88c6-1f1ce0652fb3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"57632e7d-847f-4386-a58c-a8cbc8535929","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:36Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"74217730-4c56-4b97-92f4-438ed8ec296d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"d7b48ed1-1c1f-4891-8b01-f51b52903dfb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:36Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"55ac93f4-4647-4684-8b38-db93d0df57f0","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:36Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"55ac93f4-4647-4684-8b38-db93d0df57f0","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c82beec7-b326-4e45-860e-6c38ca6788a3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:37Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a4686269-bd38-48f8-851f-9ab8f3bf1912","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:37Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0cbf5095-70fb-45b1-852a-7676d76a92ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:38Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"76a30c1a-1310-4079-89a3-a02e2b501af9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"49e3f552-b1ff-460b-8133-e9be753bf1ee","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:38Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a58adbf3-4c72-4a38-9ae9-fd0067a4f92a","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:38Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a58adbf3-4c72-4a38-9ae9-fd0067a4f92a","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c0c0929a-099a-443b-b253-25b849df5fb9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:40Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:40Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"a3f51b30-b041-4588-bf10-3cd13008de17","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:44:44Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"9b2267f9-3acc-4f93-9f66-0fb3831ebeb5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:50Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"96f4fc31-ac8b-4243-bdbb-f81fd0c7c517","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ede3c523-01d5-472e-93af-088b708c8579","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:50Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5d712093-16a1-4729-98fb-4766aa17eac9","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:50Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"5d712093-16a1-4729-98fb-4766aa17eac9","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8232be57-8354-4531-936d-0048c1ef1c31","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:57Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"202ea5ca-7f78-4cec-8765-c76f2247f666","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:44:58Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e7fe82f2-6a5b-439b-9a8b-c6cc84dcc11e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c6525509-38b7-4d92-8370-e4141a546a4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:44:58Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"bb072b82-edb7-4e6e-8014-c3093622f875","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:44:58Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"bb072b82-edb7-4e6e-8014-c3093622f875","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c55e7ab8-be04-4fc2-8b29-a1d6335032ca","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:45:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"68236961-7e34-4f05-a88b-a3c42a1b8dfe","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"bf09d3bd-4de0-4fae-8521-7e6b6ee944c4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:11Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dd4e9218-5936-4169-9b3c-2d0efda6391d","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:45:11Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"dd4e9218-5936-4169-9b3c-2d0efda6391d","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-07T07:45:21Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5bd6749b-8256-4861-bbdf-882297eda3fc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:45:31Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0f6bce00-cb7e-4017-a45e-96c9c3b71ead","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0f6bce00-cb7e-4017-a45e-96c9c3b71ead","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:31Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"08da77e8-90ef-4cab-b172-cbb4ff8ffdb2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9e5c641a-2dc2-4e20-bc8a-80a3153fb138","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:41Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a2fae406-7044-46f3-ba2e-50685a2188a0","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:45:41Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a2fae406-7044-46f3-ba2e-50685a2188a0","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"reconcile","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"apply","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.initialize"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.createDefaultGroup"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/auth.managePermissions"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:47Z","msg":"Executing action","controller":"auth","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Auth","Auth":{"name":"auth"},"namespace":"","name":"auth","reconcileID":"3036db6b-6b09-4a08-a8a2-c277f980da0a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:45:56Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ee8f16a7-345b-47b9-ada1-a29a2b1eb8f4","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:45:57Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"01d23c9a-c2b6-4068-8f6f-4be3e125d63b","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:45:57Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"01d23c9a-c2b6-4068-8f6f-4be3e125d63b","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"122e1e13-74cd-4686-9764-de40dbe1a7c6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:46:08Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"16b9fda3-5d81-430e-b5fb-9e9d1d4462e4","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"16b9fda3-5d81-430e-b5fb-9e9d1d4462e4","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:08Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"cd15aabe-286e-4e59-831e-a38f71814598","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:46:09Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:46:42Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5979850c-050a-4739-bab8-2d0963a989a6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:46:51Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"cb8ca653-3ea7-4399-8736-9b927c2a5170","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"cb8ca653-3ea7-4399-8736-9b927c2a5170","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:46:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"f29ad8b2-40cb-4309-82cd-a24e82350527","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"8c275164-fa04-42b1-9ce2-ebc635719671","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:00Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"27a85719-3e80-401d-8944-1297f94048c2","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:00Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"27a85719-3e80-401d-8944-1297f94048c2","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"60427c37-8822-4ce1-a291-f8682923e84b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:15Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1a516ab1-7c36-4bf5-b673-8c3f7ff2b943","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:15Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"1a516ab1-7c36-4bf5-b673-8c3f7ff2b943","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"722afc94-d3d9-441f-a515-c404e68708ed","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:28Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:29Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"fc06cb8d-549f-4e02-848f-020c7e3fb93d","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"fc06cb8d-549f-4e02-848f-020c7e3fb93d","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"42502dbb-e25f-4608-9978-7da99a6a0ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0ba1acd9-b34f-424a-8ab7-1d70315c132b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ad6972e1-b1b9-4599-b593-8a25764065b9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:29Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"344e7e1f-ab97-49d5-b243-ae4a09901286","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"344e7e1f-ab97-49d5-b243-ae4a09901286","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"e6672751-0504-4224-a033-dee0b11a610e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c97f066b-c1a8-41c5-8f20-ba5b078be1b5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:33Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:34Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:34Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:34Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2a1a2add-f5e9-4662-9699-5e65c2136769","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:38Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c"} {"level":"info","ts":"2026-07-07T07:47:38Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c"} {"level":"info","ts":"2026-07-07T07:47:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:38Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"7991394c-00ed-496d-be1b-355cb470ce7c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:47:39Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e968af28-5bd6-4020-a736-9befcc21cbf5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"e0657602-8da2-45cc-9324-ea8d0f985d21","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"693019af-bf21-4f21-9d6b-27ac122891bb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5e5e408a-e741-43c4-a9ce-527acd2ade79","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"1476d5c5-8cd8-4979-a348-e37a8f0f8bae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:39Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"77e0e8c1-26e3-4d82-b338-21419017e3fa","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:39Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"77e0e8c1-26e3-4d82-b338-21419017e3fa","name":"redhat-ods-applications"} {"level":"error","ts":"2026-07-07T07:47:40Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:47:41Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"c09327b7-fd01-4e98-a6f2-597f54fc1135","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:47:51Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"0b6e0bc1-41fc-4fc8-9b2e-a29600531e74","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"22dc21a6-607b-42c6-a3fc-a9260558d685","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:51Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8a5da17e-193b-47f6-939f-82b518ed5a7f","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:51Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"8a5da17e-193b-47f6-939f-82b518ed5a7f","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"48cce310-3ebc-4eb1-b445-b0ac419cc68e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:47:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8dafe63f-3596-4436-8d50-cb6139695d2a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"46757815-9f44-44c1-a57a-57e34ad99384","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"da3d5f0d-339c-44de-8338-0da4b7f05dd3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8e1a798f-8f31-4f83-802d-feac6a055004","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"82c20082-c3e6-405f-9800-480611e75c92","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:47:54Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"af28891d-0219-4892-b951-f7536cda1505","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:47:54Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"af28891d-0219-4892-b951-f7536cda1505","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:48:03Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"8726bfb8-8234-45f6-895b-842421f64658","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:06Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7238fa96-3796-4725-8150-14cb2be21173","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"9be689f5-d803-4547-920e-de0b1de3507a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:06Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"29ffda80-95a7-414f-ae03-69b8794f289e","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:48:06Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"29ffda80-95a7-414f-ae03-69b8794f289e","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"70195a88-61a8-42b7-92a6-4bb4baae8dfd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"4426f6df-786b-4d8d-a550-ff2bf541797d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:48:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"1437e5c3-216a-434f-a8e8-e00335f4050e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"309ed680-42cf-4400-a8a8-a467d1f9189e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"64aff0c6-b820-4bd8-9d22-6caf59203e1f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"212e221e-9859-41dd-97cf-4078d60cea6e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:09Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b4c58077-b947-4fd4-8b0b-28d3c11c251d","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"b4c58077-b947-4fd4-8b0b-28d3c11c251d","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ec74f1b0-ae7a-4406-850e-2b820fa3e122"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ec74f1b0-ae7a-4406-850e-2b820fa3e122"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ec74f1b0-ae7a-4406-850e-2b820fa3e122","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ec74f1b0-ae7a-4406-850e-2b820fa3e122","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ec74f1b0-ae7a-4406-850e-2b820fa3e122","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"ec74f1b0-ae7a-4406-850e-2b820fa3e122","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46"} {"level":"info","ts":"2026-07-07T07:48:09Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.initialize"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ogx.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.customizeKserveConfigMap"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a20a67b4-0a8b-4cce-9c62-41948e7f64f3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"368867db-4602-4aa9-aeed-dd5b3c212ffd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"836bd7c1-707d-42d0-988a-b02cf8ac68fb","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"d2d7fff6-2475-4b6d-ab54-65d2ed23dd46","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"3f22f142-6f41-40cd-b3c3-e94f74fd395a","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"f79ceca7-a3fd-4544-a72a-199c90f641bc","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:10Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"baf7079f-f983-481d-b2d0-be121c4e9145","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-07T07:48:11Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm","msg":"delete","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","gvk":"components.platform.opendatahub.io/v1alpha1, Kind=SparkOperator","ns":"","name":"default-sparkoperator"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"13d1ff52-8a5f-4711-b158-7c298d784661","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"ef1b54f5-a5e2-43b1-81ea-35e37c8757e7"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f2ea4db2-1953-47ca-9b8d-f5e26ef348b2"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"0e776fce-9f34-4119-bd66-c59dd0f08dd7"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"130b9ea4-51e7-47e6-8dd6-56513db7eb6e"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"cff6b099-b772-4255-a684-fe1b5480cf49"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"764c8234-8310-4288-a247-46ca6350757e"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:11Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"3e288122-7c21-4158-bde8-ac48966ed6e6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"36c9c4fb-31ac-494f-a178-8d05672fb97d"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"25e4801c-1bab-48f6-bc3b-18852e526ae3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"f5081430-b475-4d34-b0ee-210393ee12ba"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"3be86327-d421-420d-a303-f07ac7c8bb01"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"21e80d75-08dc-46d1-b141-cf3323a6c542"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"40f3e530-a0cb-44d3-9f6d-8fba778ce335"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"49d59f57-8961-44df-a3d7-e4945bb8a6b2"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"6c76d920-d9f4-4df4-885a-f2836d8ee8d4"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"20c9030a-1370-49c5-93f0-0e2e25b99921"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5581aeb1-4000-408d-bab6-6064bd63b355"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1965663c-375b-4ae5-b895-eec85925360c"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"941ea90c-948b-464d-bc60-bd2081e7a6df","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:12Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"609b006d-cf6b-4024-a225-777c75e30a28","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"609b006d-cf6b-4024-a225-777c75e30a28","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"c15a3eca-684c-4ad1-917c-38cd87fe1375","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"077f3118-f736-445f-8ca4-1e0f263b446f"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"2bc33888-f580-40fc-bf47-9b8666c7b5e5"} {"level":"info","ts":"2026-07-07T07:48:12Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"75487b16-d4b8-48a1-8d25-da2f04b7a7b7","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:48:12Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"75487b16-d4b8-48a1-8d25-da2f04b7a7b7","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"21bbfd15-a951-44bc-9465-f2b8f64c9315","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"aa8a820e-2fbb-454a-bd24-5c966b2a0de9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:48:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9eb53adb-3888-42c3-8442-3bedd5ac1dfe","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"9eb53adb-3888-42c3-8442-3bedd5ac1dfe","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5a4483da-ec96-4bec-bc30-ac2c81087b61","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"ed269438-2ee3-4a52-8ef1-89f26c5d0caf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"25fa5740-9434-44e5-8651-082e7a21d0fa","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:48:13Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:13Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"5c5eb526-8ede-4f9d-96c6-eb2054865463","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"16ed0803-bb76-4e1f-b00d-0190ffbfd27c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:14Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"7a369193-7812-4bf6-85de-1a01453b6dd5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"f9a330a9-4482-4daf-9d0a-6dc24f2a0756","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:48:14Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:48:14Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"23fa2d9a-4a41-41f9-9a92-c15edd7f2a91","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:48:15Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:48:15Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"eb4d9aab-b051-4216-b141-0aa695909fac","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:48:17Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:49:04Z","logger":"controller-runtime.cache","msg":"Warning: This version is deprecated. Use v1beta2 instead."} {"level":"error","ts":"2026-07-07T07:49:12Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"21da14bc-092e-4bed-82fe-2f77251d9907","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:29Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0a3b7d1b-c1ea-4c9c-9bc0-47152c22f049","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:49:29Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0a3b7d1b-c1ea-4c9c-9bc0-47152c22f049","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:49:47Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.initializeModules"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.cleanupDisabledModules"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.provisionModules"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/helm.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.injectModuleEnv"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/modules.updateModuleStatus"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"a498c1cd-6371-4d46-814b-b221529f405f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.deployObservabilityManifests"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modules","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"29515cd6-101a-47dd-8c3f-16650c4866f9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/dynamicownership.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/datasciencepipelines.argoWorkflowsControllersOptions"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.configureDependencies"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:49:47Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.customizeManifests"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.configureDependencies"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainingoperator.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelregistry.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelregistry","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelRegistry","ModelRegistry":{"name":"default-modelregistry"},"namespace":"","name":"default-modelregistry","reconcileID":"548396f2-a848-4264-beac-d08ea02e9ea9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/feastoperator.migrateDeploymentSelector"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"294a3cdc-b090-4094-a89b-368e7dce0113","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Pre-condition skipped by runtime predicate","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","conditionType":"LLM-D-WVADependencies"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelcontroller.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.initialize"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.removeOwnershipFromUnmanagedResources"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.cleanUpTemplatedResources"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"reconcile","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"apply","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae"} {"level":"info","ts":"2026-07-07T07:49:47Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.renderMaasOperatorInstall"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"modelcontroller","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelController","ModelController":{"name":"default-modelcontroller"},"namespace":"","name":"default-modelcontroller","reconcileID":"e8dd4aec-7b7f-4104-91c3-5debf5c4fdf6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.initialize"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trainer.checkJobSetCRD"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainingoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrainingOperator","TrainingOperator":{"name":"default-trainingoperator"},"namespace":"","name":"default-trainingoperator","reconcileID":"cf6cef77-34e9-413a-8867-ca6ac5a58e18","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a808516f-a7e9-42ed-a3d4-52e064c5bcc5"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a808516f-a7e9-42ed-a3d4-52e064c5bcc5"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"datasciencepipelines","controllerGroup":"components.platform.opendatahub.io","controllerKind":"DataSciencePipelines","DataSciencePipelines":{"name":"default-datasciencepipelines"},"namespace":"","name":"default-datasciencepipelines","reconcileID":"b38f2bc5-e917-48fe-90aa-5e2ad20a0202","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a808516f-a7e9-42ed-a3d4-52e064c5bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a808516f-a7e9-42ed-a3d4-52e064c5bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"ogx","controllerGroup":"components.platform.opendatahub.io","controllerKind":"OGX","OGX":{"name":"default-ogx"},"namespace":"","name":"default-ogx","reconcileID":"a808516f-a7e9-42ed-a3d4-52e064c5bcc5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.initialize"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/mlflowoperator.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trainer","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Trainer","Trainer":{"name":"default-trainer"},"namespace":"","name":"default-trainer","reconcileID":"feeb29c6-8253-41c2-91d8-fd976666dad9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0e563cf4-0118-4521-9578-e72497abc962","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"0e563cf4-0118-4521-9578-e72497abc962","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.initialize"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.configureDependencies"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.setKustomizedParams"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.initialize"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/trustyai.createConfigMap"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/sanitycheck.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"11f0cd8e-6c8c-43f6-a42a-140004e1e4e5","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"trustyai","controllerGroup":"components.platform.opendatahub.io","controllerKind":"TrustyAI","TrustyAI":{"name":"default-trustyai"},"namespace":"","name":"default-trustyai","reconcileID":"2778d8a5-aab8-4954-aeb4-1b9112e3aad0","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"mlflowoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"MLflowOperator","MLflowOperator":{"name":"default-mlflowoperator"},"namespace":"","name":"default-mlflowoperator","reconcileID":"7598c633-ab7d-47b8-a34e-574271f069d7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"feastoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"FeastOperator","FeastOperator":{"name":"default-feastoperator"},"namespace":"","name":"default-feastoperator","reconcileID":"2c084afe-d725-48b7-8e70-5a526ebdc862","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/modelsasservice.ensureMaasClusterConfigControllerRef"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"modelsasservice","controllerGroup":"components.platform.opendatahub.io","controllerKind":"ModelsAsService","ModelsAsService":{"name":"default-modelsasservice"},"namespace":"","name":"default-modelsasservice","reconcileID":"c1c033c8-76b0-473e-ab03-b357a41977ae","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kserve.reconcileModelCache"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/ray.initialize"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"kserve","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kserve","Kserve":{"name":"default-kserve"},"namespace":"","name":"default-kserve","reconcileID":"8eb7e32d-c0e4-4a4e-9aab-ed78ad0e5d86","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:49Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:50Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"ray","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Ray","Ray":{"name":"default-ray"},"namespace":"","name":"default-ray","reconcileID":"0358154c-94c8-4e5f-9962-30a7cc61e9cd","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"e60c7491-1679-42fc-8f28-7b22dc9d106e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:51Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:49:51Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"71a7ce2e-40e0-4706-917f-b2b0b10fecab","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"71a7ce2e-40e0-4706-917f-b2b0b10fecab","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:49:51Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"4243a726-16fc-40a5-ae3d-917de2c03633","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/imagestreams.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/workbenches.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"c9ad6bde-63b0-4cb5-9b31-d22786ed3790","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"c9ad6bde-63b0-4cb5-9b31-d22786ed3790","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"d0fd0eb9-cc35-44d5-8e93-28b25e22188e","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:49:52Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"5c956933-67ca-4d12-84f4-d4a8e36c97d6","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:52Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a414a067-b157-49d6-8c9f-380b997bdb22","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:49:52Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"a414a067-b157-49d6-8c9f-380b997bdb22","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.reconcileHardwareProfiles"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/dashboard.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"dashboard","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Dashboard","Dashboard":{"name":"default-dashboard"},"namespace":"","name":"default-dashboard","reconcileID":"ef82db5e-6062-4763-bd47-a6c9fcce5ac3","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"reconcile","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"apply","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createGatewayInfrastructure"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createKubeAuthProxyInfrastructure"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createEnvoyFilter"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createNetworkPolicy"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createOCPRoutes"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects"} {"level":"info","ts":"2026-07-07T07:49:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.createDashboardRedirects.createDashboardRedirects","msg":"Creating dashboard redirect resources","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","dashboardRouteName":"rhods-dashboard","namespace":"redhat-ods-applications","currentSubdomain":"rh-ai"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/gateway.syncGatewayConfigStatus"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"gatewayconfig","controllerGroup":"services.platform.opendatahub.io","controllerKind":"GatewayConfig","GatewayConfig":{"name":"default-gateway"},"namespace":"","name":"default-gateway","reconcileID":"eea5bc28-c0a8-4457-a732-c89cf8f04b34","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"e05d508a-c7e4-4a8b-b76e-26d4696ef184","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:53Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"684be73b-91f6-49e8-b542-2c73176042b1","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"workbenches","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Workbenches","Workbenches":{"name":"default-workbenches"},"namespace":"","name":"default-workbenches","reconcileID":"42b94810-07ca-48d3-8f73-aa5c68f704da","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"5e363f8e-324c-429c-a9a3-61345e42e3a7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:53Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"dfe62712-a0d6-4223-930e-058f87f63ebf","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:49:54Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:49:54Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"adb00fb1-5f27-4d0d-80c0-5715dec4ec4c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:50:02Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"14f2fd73-ad6f-4dea-a815-e39d5c8501e9","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:50:05Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"a742c582-a79f-425d-bcfd-196949d688be","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"618e2b09-d02f-4f58-910f-aaa8fb6cd34d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"reconcile","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"apply","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.addMonitoringCapability"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringAdmissionPolicies"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployMonitoringStackWithQuerierAndRestrictions"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployTracingStack"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployAlerting"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployOpenTelemetryCollector"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPerses"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesTempoIntegration"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployPersesPrometheusIntegration"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.deployNodeMetricsEndpoint"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/template.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/services/monitoring.syncPrometheusWebTLSCA"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Executing action","controller":"monitoring","controllerGroup":"services.platform.opendatahub.io","controllerKind":"Monitoring","Monitoring":{"name":"default-monitoring"},"namespace":"","name":"default-monitoring","reconcileID":"19046240-f726-454c-a7b4-8d58e597d601","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:50:05Z","logger":"DSCInitialization","msg":"Reconciling DSCInitialization.","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"54902f57-4c4f-4f75-a7f2-e77041f03688","DSCInitialization Request.Name":"default-dsci"} {"level":"info","ts":"2026-07-07T07:50:05Z","msg":"Application namespace set in DSCI not found, creating it with labels","controller":"dscinitialization","controllerGroup":"dscinitialization.opendatahub.io","controllerKind":"DSCInitialization","DSCInitialization":{"name":"default-dsci"},"namespace":"","name":"default-dsci","reconcileID":"54902f57-4c4f-4f75-a7f2-e77041f03688","name":"redhat-ods-applications"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"1928a2be-95b7-4421-821e-59a259ea237f","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"reconcile","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"apply","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/sparkoperator.initialize"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:50:09Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"sparkoperator","controllerGroup":"components.platform.opendatahub.io","controllerKind":"SparkOperator","SparkOperator":{"name":"default-sparkoperator"},"namespace":"","name":"default-sparkoperator","reconcileID":"81467b48-d012-4f9b-b2c4-a593ac88eb3b","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:50:09Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"ed7a3ccd-1d36-4359-9359-f496f3ac7d1d","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:51:00Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"kuadrant-system","reconcileID":"081200eb-6df7-418a-b477-4dad2ed8ce42"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d924c9dl1jkc7bfof6k0","reconcileID":"a984f4b0-9253-46b7-bab3-f06d8eb77248"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d952vbkc1s0s7b3oe5f0","reconcileID":"cbb61d1b-1e49-495b-943d-0ac20907ffe7"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95ah4rkn3vc74rcg6sg","reconcileID":"31e0aa0b-a7de-4ae4-9e8b-a39dd91f8030"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95eh6ktemqs7bcktd70","reconcileID":"0793d1fc-10ba-42ae-af8f-186b529fce91"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d969ai388e3s7b17337g","reconcileID":"40f1cae2-4dd6-46d6-88e3-c8d3495694e5"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"openldap","reconcileID":"8174d65e-223e-4240-8408-7723c5e911be"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"this-is-a-long-namespace-name","reconcileID":"81c1e65e-5ab0-4101-8d92-3911d3bb862a"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"workload-variant-autoscaler-system","reconcileID":"a0ebeb5f-b3b4-49a5-8956-1225cd47f1dd"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92efpig05uc74vdbpfg","reconcileID":"c184fe67-5ca4-4be3-ade4-7637eba7863d"} {"level":"info","ts":"2026-07-07T07:51:17Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d931ib323dac7b8v1iag","reconcileID":"85ce37c4-ad7e-4cde-b4cf-1c33bcae59b3"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93aruo495mc7b420ucg","reconcileID":"7b1e84d1-1922-4c74-837c-900d94673f80"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95lve148los74uh76d0","reconcileID":"0e7df9c8-b77e-4eb8-a253-b519dd2fb2a5"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d91ni73pd9bs7bavp5l0","reconcileID":"4ca3c8bc-489f-4f43-a8f1-a2a3bbe76fa1"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d924bsll1jkc7bfof6jg","reconcileID":"ddd19a26-83be-4bc5-9882-bc636f25025c"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92hftr6a04c7b765c7g","reconcileID":"773aa8c7-65a9-46b8-9b0e-db636544dfb2"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d955it4c1s0s7b7c753g","reconcileID":"2fea99fc-b9fe-494c-a2af-52df934ba48e"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d955j9sc1s0s7b7c7540","reconcileID":"74537ac8-8ad3-4818-ba75-66258ecc2b83"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d92hgar6a04c7b765c80","reconcileID":"63725ee3-2d8e-4284-b066-a2609d8148da"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d957q506ngts7b1qap80","reconcileID":"25cad6a1-ee73-41e0-938a-cd14d9237704"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d92eggag05uc74vdbpg0","reconcileID":"8c963f6d-41e5-4090-a189-9b126f9c4bb1"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cert-manager","reconcileID":"bf5f0cc0-ff87-466b-91e3-f88e0a9ea0da"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"rhoai-model-registries","reconcileID":"21b2cd8c-ae61-4fd1-a33a-045c795b646c"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d91unc9qm98c7b5mhqlg","reconcileID":"308d62a9-7eb8-48f1-a895-92fb32c51e2f"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93ov2pomnac7b55np0g","reconcileID":"aab9f6aa-52e1-42b3-88c1-d5163b8f97d0"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95cvfg7cpoc7bbvpiog","reconcileID":"ec684ddb-de6c-41bb-80e5-b08fa7940120"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d95d01g7cpoc7bbvpipg","reconcileID":"28f109e8-dafb-4975-abdf-0fe433c431e3"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-dw-ui-proj-73263","reconcileID":"d503977e-0e4f-4df2-b257-397f105e0b8b"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d924bndl1jkc7bfof6j0","reconcileID":"f0d519d4-aee9-4821-8a92-0e084b7d678b"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92hfob6a04c7b765c70","reconcileID":"f914255e-e222-4be5-84f8-6eb1ffa0a2f6"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d931igb23dac7b8v1ib0","reconcileID":"87df1b58-04d4-498d-a1c0-3d0aeb79ce4c"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d93dbj0tiems74rspei0","reconcileID":"153def76-795d-4ced-b4db-280a9c967b5f"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d93ovfhomnac7b55np10","reconcileID":"5f4d7ae1-1101-4a9f-807e-ba2b972f0486"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93u89rg324s7b2qcieg","reconcileID":"4883a01c-b21f-40b6-b734-cd8ff3a0c75d"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d95ehoktemqs7bcktd80","reconcileID":"8c964306-9efb-4a3c-9344-43b496f4b1ba"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"redhat-ods-monitoring","reconcileID":"2a8acc66-bea5-450b-8cee-3a3dee064266"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92efkag05uc74vdbpf0","reconcileID":"2006d2a6-b41a-469b-b4fb-2f8332bb5e5e"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92ju8emdfuc74ubedr0","reconcileID":"affca4bb-d382-46c0-b398-0d28aeee52cf"} {"level":"info","ts":"2026-07-07T07:51:18Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93arpg495mc7b420uc0","reconcileID":"65c9c68d-bf1c-459a-a1c7-efbdceb33105"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d950876c0qgc7b422fig","reconcileID":"8489719f-c7e3-4f56-b719-23a97f5b5faa"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d952uusc1s0s7b3oe5eg","reconcileID":"0530b88b-15ce-4c43-983f-f34d7c3c2864"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95lv8p48los74uh76cg","reconcileID":"ac9c1bc2-8a66-4116-b4c2-83c441ccd60a"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95p8fvdpamc7b5eelu0","reconcileID":"9a631d3a-b2e6-42de-a7f4-99054467a6aa"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"redhat-ods-applications","reconcileID":"e030c6a7-381f-4028-9132-8e00723dd681"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"redhat-ods-operator","reconcileID":"d73c0f7b-f71b-44a0-9903-5d8a5006c4dc"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93outhomnac7b55np00","reconcileID":"d36f08f3-3503-45bc-a08d-cea997a9e2d3"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d957qmo6ngts7b1qap90","reconcileID":"f63596ab-b03c-45e3-a92a-f5caee24f795"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d95lvqp48los74uh76dg","reconcileID":"c1f86bc3-9111-4378-9a95-710468aff856"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d95p91vdpamc7b5eelv0","reconcileID":"dd6659a1-8331-4e23-a596-b9ba3db040a4"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"models-as-a-service","reconcileID":"d3d0175c-60e4-4b82-b98d-b8c5d09e7a18"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93u84rg324s7b2qcie0","reconcileID":"f977aa18-ffcd-495a-a079-830339f50f21"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d9508juc0qgc7b422fj0","reconcileID":"b9dc96af-e51c-404c-931d-32e4316a570d"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"ai-tenants","reconcileID":"a8aa919b-2f19-4841-92f6-6dea03e0229b"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"anthony-upgrade-project","reconcileID":"24340153-f11c-4991-ab00-f0d275567d0e"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"rhods-notebooks","reconcileID":"146d27dd-7c69-4c5f-84c6-62dbe9271775"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d91uo8pqm98c7b5mhqmg","reconcileID":"51ee1d1b-cf74-437c-b607-e2e9a20e5900"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92njve4r74c7b6oturg","reconcileID":"5b13eac7-670a-4019-942b-5caccc764a1d"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d935dmsn4eus7b2dci8g","reconcileID":"3da61370-d52e-46bd-8ca5-c10b967a33bc"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d93asbg495mc7b420ud0","reconcileID":"b27e39c3-13bd-4045-813c-b534c5dfab04"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d955insc1s0s7b7c7530","reconcileID":"de566fcf-c09f-4142-8016-b5d90e42042a"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93db60tiems74rspehg","reconcileID":"82d8fa52-44f3-4837-ab51-e255d3f13d22"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95cvko7cpoc7bbvpip0","reconcileID":"83de278a-dfc7-4950-b588-7fc7633ad90b"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95p8l7dpamc7b5eelug","reconcileID":"a9c75a5f-1229-4dc0-804f-939eed433e24"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d935e3sn4eus7b2dci90","reconcileID":"69e45b4e-f89a-4275-b9c4-b5f607a2ba8c"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"cert-manager-operator","reconcileID":"51ecb34c-14de-411f-847d-35df6e79f202"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92ju2umdfuc74ubedqg","reconcileID":"9baf463d-9fb6-4fd6-84f9-fc6182e597d5"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d92juvmmdfuc74ubedrg","reconcileID":"fc9185bd-af7b-47e9-8e61-1a8f1226c7f6"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d92nkhm4r74c7b6otusg","reconcileID":"4eed3e58-d802-413a-b80d-8ae99b8976db"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d957qa86ngts7b1qap8g","reconcileID":"34e9ee22-15b2-467a-b869-39c2766999b8"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d95ahmrkn3vc74rcg6tg","reconcileID":"96628be3-bf49-42bc-94dc-32f5e3a9c80a"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95ehbstemqs7bcktd7g","reconcileID":"fef80416-b135-4981-a64b-03a190ad49e5"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d931itb23dac7b8v1ibg","reconcileID":"841bacdb-b3e0-4e4e-a893-26b089fc9bde"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d969anb88e3s7b173380","reconcileID":"9051a0dc-8a85-4e64-82a8-758ac84a0c10"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"aaa-fede","reconcileID":"193fcc39-15d4-458e-93f5-26e252793df3"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d91unhpqm98c7b5mhqm0","reconcileID":"0008d41b-758c-4fc2-b8b6-27d8891501c9"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d93db0otiems74rspeh0","reconcileID":"5caedeba-6ae4-43da-b7ff-925d5a939dc0"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d93u8mrg324s7b2qcif0","reconcileID":"5829a486-aed7-4956-81d3-73939b368cfe"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d94mg9lhpi1c7b5d0v5g","reconcileID":"03a32c4f-9fcf-49bf-9a77-447190398c9f"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d94mgrthpi1c7b5d0v6g","reconcileID":"b500074b-2689-4720-99d8-a53d033f6664"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d952upkc1s0s7b3oe5e0","reconcileID":"257f3d64-1350-44ee-904d-a7946a287cb4"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-no-queues-d969b4b88e3s7b17338g","reconcileID":"64c93683-5f1c-4ae4-9d08-3e7f2745e650"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d92nk4m4r74c7b6otus0","reconcileID":"c78f2501-bf99-425b-82a3-a7b24955a181"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d935dhkn4eus7b2dci80","reconcileID":"bf9a63fd-2636-4ce1-9f63-002f18d723fa"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d94mgf5hpi1c7b5d0v60","reconcileID":"71afd3ee-f72d-427f-96b0-6e6f875ac59a"} {"level":"info","ts":"2026-07-07T07:51:19Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95081uc0qgc7b422fi0","reconcileID":"cb46a95c-3deb-46ac-9203-fe274400048e"} {"level":"info","ts":"2026-07-07T07:51:20Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-kueue-managed-d95aha3kn3vc74rcg6t0","reconcileID":"111c6b20-dcb1-4297-a9cf-c6a423a27eac"} {"level":"error","ts":"2026-07-07T07:51:36Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"info","ts":"2026-07-07T07:51:42Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-90xzs","reconcileID":"c5813627-f411-441d-bf93-960241f521e0"} {"level":"info","ts":"2026-07-07T07:51:42Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-90xzs","reconcileID":"f742e700-8c61-4c27-8d10-c40be952e447"} {"level":"info","ts":"2026-07-07T07:51:42Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-90xzs","reconcileID":"cdb3f030-a128-4229-933c-d6ed2a5e33a1"} {"level":"info","ts":"2026-07-07T07:51:43Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-i71hg","reconcileID":"d42e766c-9b9a-4319-a3ea-d5e3f5438478"} {"level":"info","ts":"2026-07-07T07:51:43Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-i71hg","reconcileID":"4f600264-91ab-41a0-9fec-21ef065f0175"} {"level":"info","ts":"2026-07-07T07:51:43Z","msg":"Adding CA bundle configmap","controller":"cert-configmap-generator-controller","namespace":"","name":"test-ns-i71hg","reconcileID":"bd09f526-334e-43bc-ac3c-b75d9ba2c14f"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:46Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:51:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:51:47Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:51:47Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"44674a06-2783-4456-b1eb-1fcf5287a2b2","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"reconcile","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"apply","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.initialize"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/releases.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/render/kustomize.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageDefaultKueueResourcesAction"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:51:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"2294f066-7012-4437-bf83-88efc1af6cb7","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding"} {"level":"info","ts":"2026-07-07T07:51:48Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.manageKueueAdminRoleBinding","msg":"Successfully managed kueue admin role binding","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","adminGroups":["rhods-admins"]} {"level":"info","ts":"2026-07-07T07:51:48Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/status/deployments.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.(*componentHandler).NewComponentReconciler.func1"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/components/kueue.configureClusterQueueViewerRoleAction"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"kueue","controllerGroup":"components.platform.opendatahub.io","controllerKind":"Kueue","Kueue":{"name":"default-kueue"},"namespace":"","name":"default-kueue","reconcileID":"4482e749-8375-4186-ba08-bb164d2ed831","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"reconcile","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"apply","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.initialize"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.checkPreConditions"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus"} {"level":"info","ts":"2026-07-07T07:51:49Z","logger":"action.github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.updateStatus.checkMaaSPrerequisites","msg":"MaaS gateway missing required annotations","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","gateway":"maas-default-gateway","missing":["security.opendatahub.io/authorino-tls-bootstrap=\"true\""]} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/internal/controller/datasciencecluster.provisionComponents"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/deploy.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/actions/gc.(*Action).run-fm"} {"level":"info","ts":"2026-07-07T07:51:49Z","msg":"Executing action","controller":"datasciencecluster","controllerGroup":"datasciencecluster.opendatahub.io","controllerKind":"DataScienceCluster","DataScienceCluster":{"name":"default-dsc"},"namespace":"","name":"default-dsc","reconcileID":"5d5e0464-9b3e-456c-b961-720cf6e16b8c","action":"github.com/opendatahub-io/opendatahub-operator/v2/pkg/controller/reconciler.(*dynamicWatchAction).run-fm"} {"level":"error","ts":"2026-07-07T07:52:31Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} {"level":"error","ts":"2026-07-07T07:53:10Z","logger":"controller-runtime.cache.UnhandledError","msg":"Failed to watch","reflector":"pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:289","type":"operator.authorino.kuadrant.io/v1beta1, Kind=Authorino","error":"authorinos.operator.authorino.kuadrant.io is forbidden: User \"system:serviceaccount:redhat-ods-operator:redhat-ods-operator-controller-manager\" cannot watch resource \"authorinos\" in API group \"operator.authorino.kuadrant.io\" at the cluster scope","stacktrace":"k8s.io/apimachinery/pkg/util/runtime.logError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:221\nk8s.io/apimachinery/pkg/util/runtime.handleError\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:212\nk8s.io/apimachinery/pkg/util/runtime.HandleErrorWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/runtime/runtime.go:198\nk8s.io/client-go/tools/cache.DefaultWatchErrorHandler\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:204\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:370\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:255\nk8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:256\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/backoff.go:233\nk8s.io/client-go/tools/cache.(*Reflector).RunWithContext\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/client-go@v0.35.3/tools/cache/reflector.go:368\nk8s.io/client-go/tools/cache.(*controller).RunWithContext.(*Group).StartWithContext.func3\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:63\nk8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1\n\t/cachi2/output/deps/gomod/pkg/mod/k8s.io/apimachinery@v0.35.3/pkg/util/wait/wait.go:72"} ==================== OPERATOR POD LOG: rhods-operator-6f6bb89fdf-f4zpw.log ==================== ==================== OPERATOR POD LOG: rhods-operator-6f6bb89fdf-kr2pd.log ==================== ==================== WORKLOAD POD LOG: redhat-ods-applications/dashboard-redirect-6bd46884d4-6mllj.log ==================== 2026/07/07 06:19:16 [notice] 1#1: using the "epoll" event method 2026/07/07 06:19:16 [notice] 1#1: nginx/1.26.3 2026/07/07 06:19:16 [notice] 1#1: built by gcc 11.5.0 20240719 (Red Hat 11.5.0-14) (GCC) 2026/07/07 06:19:16 [notice] 1#1: OS: Linux 5.14.0-570.60.1.el9_6.x86_64 2026/07/07 06:19:16 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576 2026/07/07 06:19:16 [notice] 1#1: start worker processes 2026/07/07 06:19:16 [notice] 1#1: start worker process 12 2026/07/07 06:19:16 [notice] 1#1: start worker process 13 ==================== WORKLOAD POD LOG: redhat-ods-applications/data-science-pipelines-operator-controller-manager-7fc7f76tthhm.log ==================== 2026-07-07T06:19:21Z INFO setup starting manager 2026-07-07T06:19:21Z INFO controller-runtime.metrics Starting metrics server 2026-07-07T06:19:21Z INFO starting server {"name": "health probe", "addr": "[::]:8081"} 2026-07-07T06:19:21Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false} I0707 06:19:21.186758 1 leaderelection.go:258] "Attempting to acquire leader lease..." lock="redhat-ods-applications/f9eb95d5.opendatahub.io" I0707 06:20:35.892209 1 leaderelection.go:272] "Successfully acquired lease" lock="redhat-ods-applications/f9eb95d5.opendatahub.io" 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Deployment"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Secret"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Secret"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.DataSciencePipelinesApplication"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.ConfigMap"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Service"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.ServiceAccount"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.ConfigMap"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.PersistentVolumeClaim"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Pod"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.NetworkPolicy"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Role"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.RoleBinding"} 2026-07-07T06:20:35Z INFO Starting EventSource {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "source": "kind source: *v1.Route"} 2026-07-07T06:20:36Z INFO Starting Controller {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication"} 2026-07-07T06:20:36Z INFO Starting workers {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "worker count": 10} 2026-07-07T06:31:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:03Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:03Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:03Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:03Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:04Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:04Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:04Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:04Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:04Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:04Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:06Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:06Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:06Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:06Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:06Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:06Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:06Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:06Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:13Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:13Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:13Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:13Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:13Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:13Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:13Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:13Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:14Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:14Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:14Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:14Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:14Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Unable to connect to Database: dial tcp 172.30.201.185:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:14Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:14Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:14Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:14Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:14Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:31:25Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:25Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:26Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:27Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:27Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:27Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:27Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:27Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:27Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:27Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:27Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:27Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:28Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:28Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:28Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:28Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:28Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:28Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:28Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:28Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:28Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:29Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:29Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:29Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:30Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:30Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:30Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:30Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:30Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:30Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:30Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:30Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:31Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:31Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:31Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:31Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:31Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:31Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:31Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:31Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:32Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:32Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:32Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:32Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:32Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:32Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:32Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:32Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:34Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:34Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:34Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:34Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:34Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:31:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:40Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:31:41Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:31:41Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:22Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:22Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:22Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:22Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "d558e22d-2342-4cf5-93cb-5690212524bf", "error": "services \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "327e9fcf-ffb9-4b65-8a01-e3f14a0b53c2", "error": "services \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "23bb9622-dd5e-4815-9c51-b2f3bbd41796", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "09b3dda3-9cb4-4474-a1bf-d2e12f2ac92a", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "72cfebcc-e23c-4030-9b57-0b004a020de6", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "858c9a39-e813-4814-9dcf-9dd72db75621", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "83ca6156-6cbb-4da7-8fd0-d6e9d93c0d0e", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:23Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:23Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:23Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "040f4caf-21ff-4378-b3ea-01e8cd787f9a", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "object storage password from secret [dashboard-dspa-secret] for keys [AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY] was not successfully retrieved, ensure that the secret with this key exist"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:32:24Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:32:24Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:32:24Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:32:24Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:32:25Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:32:25Z INFO DSPA resources cleaned up. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:04Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:04Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:04Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:04Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:04Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:04Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:04Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:04Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:05Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:05Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:05Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:05Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:06Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:06Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:06Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:06Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:06Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:06Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:06Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:06Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:07Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:07Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:07Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:07Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:07Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:07Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:07Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:07Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:17Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:17Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:17Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Unable to connect to Database: dial tcp 172.30.15.192:3306: connect: connection refused {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:17Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:17Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:17Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:33:17Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:28Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:28Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:29Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:29Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:29Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:29Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:30Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:30Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:30Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:30Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:30Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:30Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:30Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:30Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:31Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:31Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:31Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:31Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:31Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:31Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:32Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:32Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:33Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:33Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:33Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:33Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:33Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:33Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:33Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:33Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:35Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:36Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:37Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:38Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:38Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:39Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:39Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:39Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:33:39Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:39Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:42Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying Database Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying storage Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Performing Database Health Check {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Database Health Check Successful {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying Common Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Generating Sample Config {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying APIServer Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying PersistenceAgent Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying ScheduledWorkflow Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying WorkflowController Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Applying ML-Metadata (MLMD) Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Finished applying MLMD Resources {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:33:43Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:33:43Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "True"} 2026-07-07T06:34:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:35Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:35Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:35Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:35Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace cy-e2e-automl-binary-522120, this bundle will be included in external tls connections. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "c3275eea-05b8-490c-9d48-6f5ec0be6661", "error": "networkpolicies.networking.k8s.io \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "0a713c84-fb41-4432-b0de-eebe46435b8b", "error": "networkpolicies.networking.k8s.io \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "01a77bbc-bafc-4419-8723-d4778917f295", "error": "networkpolicies.networking.k8s.io \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "663d66eb-3365-4cff-b052-e7bba8904675", "error": "networkpolicies.networking.k8s.io \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "fb6bab31-6c74-4f46-a799-51d1f04bf5ad", "error": "networkpolicies.networking.k8s.io \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "e8a5a99b-defd-4b7e-8df4-75d151217e0f", "error": "networkpolicies.networking.k8s.io \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:36Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:36Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:36Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "dee86be1-0ffd-4d44-9e8e-55ba9d5b80c7", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "f9910894-b04c-4e55-ba62-f0ced6d0e428", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "8cb283dd-aa49-44a9-a598-2f3ade8554a0", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "856b42d8-47ce-4c84-b058-46c6dc42177d", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "106649ae-2a9a-408c-9a50-b22c17bcb95f", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "7865b8e6-2d90-4134-9bae-18a5866bab0c", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Applying mariaDB resources. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Publishing Ready Metrics {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO ObjectStoreAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO APIServerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO PersistenceAgentReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO ScheduledWorkflowReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO WorkflowControllerReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z INFO Ready {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO DatabaseAvailable {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:34:37Z INFO MLMDProxyReady {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:34:37Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"cy-e2e-automl-binary-522120"}, "namespace": "cy-e2e-automl-binary-522120", "name": "dspa", "reconcileID": "0e48d8ed-7132-47ad-9e53-cf0e62da9284", "error": "deployments.apps \"mariadb-dspa\" is forbidden: unable to create new content in namespace cy-e2e-automl-binary-522120 because it is being terminated"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:34:37Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:34:37Z INFO DSPA resources cleaned up. {"namespace": "cy-e2e-automl-binary-522120", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:32Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:32Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:32Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:32Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:33Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:33Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:33Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:33Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:34Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:34Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:34Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:34Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:34Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:34Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:34Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:34Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:34Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:35Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:35Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:35Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:35Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:37Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:37Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:48Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:49Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:49Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:49Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:49Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:42:49Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:42:49Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:49Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:49Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:42:49Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:42:49Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:43:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:43:29Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:29Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:29Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:29Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:29Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:29Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:29Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:29Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:29Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:49Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:43:49Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:43:49Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:49Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:43:49Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:49Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:49Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:49Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:49Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:43:49Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:44:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:44:29Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:29Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:29Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:29Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:29Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:29Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:29Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:29Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:29Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:49Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:44:49Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:44:49Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:49Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:49Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:49Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:49Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:49Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:44:49Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:44:49Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:09Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:45:09Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:09Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:09Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:09Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:09Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:09Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:09Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:09Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:09Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:29Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:29Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:45:30Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:30Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:30Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:30Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:30Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:30Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:30Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:30Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:30Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:30Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:50Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:45:50Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:45:50Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:50Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:50Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:50Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:50Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:45:50Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:50Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:45:50Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:46:10Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:10Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:10Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:10Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:10Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:10Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:10Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:10Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:10Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:46:30Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:30Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:30Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:30Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:30Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:30Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:30Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:30Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:30Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:50Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:46:50Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:46:50Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:50Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:50Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:50Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:50Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:46:50Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:50Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:46:50Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:47:10Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:10Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:10Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:10Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:10Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:10Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:10Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:10Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:10Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:47:30Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:30Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:30Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:30Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:30Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:30Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:30Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:30Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:30Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:50Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:47:50Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:47:50Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:50Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:50Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:50Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:50Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:47:50Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:50Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:47:50Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:48:10Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:10Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:10Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:10Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:10Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:10Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:10Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:10Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:10Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:30Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:48:30Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:30Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:30Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:30Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:30Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:30Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:30Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:30Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:30Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:50Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:48:50Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:48:50Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:50Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:50Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:50Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:50Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:50Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:48:50Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:48:50Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:10Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:49:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:49:31Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:31Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:31Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:31Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:31Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:31Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:31Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:31Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:31Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:51Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:49:51Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:49:51Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:51Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:51Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:51Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:51Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:49:51Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:51Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:49:51Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:11Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:11Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:11Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:11Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:11Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:11Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:11Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:11Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:11Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:11Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:31Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:31Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:31Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:31Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:31Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:31Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:31Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:31Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:31Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:31Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:37Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:37Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:37Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:37Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:50:37Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:50:37Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:43Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:43Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:44Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:44Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:44Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:44Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:45Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:45Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:45Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:45Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:45Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:45Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:45Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:48Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:48Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:48Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:48Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:48Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:48Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:48Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:48Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:48Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:59Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:50:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:50:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:59Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:50:59Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:50:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:19Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:19Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:19Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:19Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:51:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:20Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:51:20Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:20Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:20Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:51:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:51:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:51:40Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:51:40Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:40Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:51:40Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:40Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:40Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:40Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:51:40Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:51:40Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:52:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:52:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:20Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:20Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:20Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:20Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:52:40Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:52:40Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:40Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:40Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:40Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:40Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:40Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:52:40Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:52:40Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:00Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:00Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:20Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:20Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:20Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:20Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:40Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:40Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:40Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:40Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:40Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:40Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:40Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:40Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:41Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:53:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:53:41Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:53:41Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:41Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:53:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:54:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:01Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:01Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:01Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:01Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:54:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:54:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:54:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:21Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:21Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:54:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:54:41Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:54:41Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:41Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:54:41Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:55:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:01Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:01Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:01Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:01Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:55:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:22Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:22Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:22Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:22Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:55:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:55:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:42Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:42Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:42Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:55:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:55:42Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:56:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:02Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:02Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:02Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:02Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:56:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:22Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:22Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:22Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:22Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:56:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:56:42Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:42Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:42Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:56:42Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:56:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:57:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:02Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:02Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:02Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:02Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:57:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:22Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:22Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:22Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:22Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:57:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:42Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:57:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:57:42Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:42Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:57:42Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:57:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:02Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:02Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:02Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:02Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:03Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:23Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:23Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Performing Database Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:23Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:23Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:23Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-custom-pip-prj-150747, this bundle will be included in external tls connections. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-custom-pip-prj-150747 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:36Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:36Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:36Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:36Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:37Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:37Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:37Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:58:37Z INFO Publishing Ready Metrics {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:58:37Z INFO DatabaseAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO APIServerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:37Z INFO ObjectStoreAvailable {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO PersistenceAgentReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO WorkflowControllerReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO MLMDProxyReady {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:58:37Z INFO Ready {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:58:37Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:58:37Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T06:58:37Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-custom-pip-prj-150747", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:37Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:37Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:37Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:37Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:37Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:37Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:38Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:38Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:38Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:38Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:38Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:38Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:38Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:38Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:38Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:39Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:39Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:39Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:39Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:39Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:39Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:39Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:39Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:40Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:40Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:40Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:40Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:40Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:40Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:40Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:40Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:40Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:48Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:48Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:48Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:48Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:48Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:48Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:48Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:48Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:48Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:49Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:49Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:49Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:49Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T06:59:49Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T06:59:49Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:49Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:49Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T06:59:49Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T06:59:49Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:00:00Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:00Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:00Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:00Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:00Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:00Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:00Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:00Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:00:01Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:00:21Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:21Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:21Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:21Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:21Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:21Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:21Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:21Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:21Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:00:41Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:00:41Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:41Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:41Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:41Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:41Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:00:41Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:41Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:00:41Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:01:01Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:01:01Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:01:01Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:01:21Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:21Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:21Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:21Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:21Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:21Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:21Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:21Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:21Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:01:41Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:01:41Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:41Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:41Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:41Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:41Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:01:41Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:41Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:01:41Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:02:01Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:01Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:01Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:01Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:01Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:01Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:01Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:01Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:01Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:02:21Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:21Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:21Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:21Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:02:21Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:21Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:21Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:21Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:21Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:21Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:02:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:02:42Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:02:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:02:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:02:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:03:02Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:03:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:03:42Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:03:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:03:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:03:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:04:02Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:02Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:02Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:02Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:02Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:02Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:02Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:02Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:02Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:04:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:04:22Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:22Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:22Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:04:23Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:23Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:23Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:23Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:23Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:23Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:23Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:23Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:23Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:04:43Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:04:43Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:43Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:43Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:43Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:04:43Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:43Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:43Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:04:43Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:05:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:05:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:05:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:05:23Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:23Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:23Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:23Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:23Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:23Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:23Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:23Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:23Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:05:43Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:05:43Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:43Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:43Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:43Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:43Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:05:43Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:43Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:05:43Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:06:03Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:03Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:03Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:03Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:03Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:03Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:03Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:03Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:03Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:06:23Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:23Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:23Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:23Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:23Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:23Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:23Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:23Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:23Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:06:44Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:06:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:06:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:06:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:04Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:04Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:04Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:04Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:04Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:04Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:04Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:04Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:04Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:24Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:24Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:24Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:24Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:24Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:24Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:24Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:24Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:24Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:42Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:42Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:42Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:42Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:42Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:43Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:43Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:43Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:43Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:43Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:43Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:43Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:43Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:43Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:43Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:44Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:44Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:44Z ERROR Encountered error when updating the DSPA status {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "Operation cannot be fulfilled on datasciencepipelinesapplications.datasciencepipelinesapplications.opendatahub.io \"dspa\": the object has been modified; please apply your changes to the latest version and try again"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).updateStatus /workspace/controllers/dspipeline_controller.go:595 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:453 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:44Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO DSPA resources cleaned up. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:44Z INFO DataSciencePipelinesApplication.datasciencepipelinesapplications.opendatahub.io "dspa" not found {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:52Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:52Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:52Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:53Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:53Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:53Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:53Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:53Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:53Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:53Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:53Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:53Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:56Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:56Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:56Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:56Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:56Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:56Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:56Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:56Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:56Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:56Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:07:59Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:07:59Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:59Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:59Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:59Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:07:59Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:59Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:59Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:59Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:07:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:00Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:00Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:00Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:00Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:00Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:00Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:12Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:12Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:12Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:12Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:12Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:12Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:12Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:12Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:12Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:12Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:13Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:13Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:13Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:13Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:13Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:13Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:13Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:13Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:13Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:08:53Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:08:53Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:53Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:53Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:53Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:53Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:53Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:08:53Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:08:53Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:09:13Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:13Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:13Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:13Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:13Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:13Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:13Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:13Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:13Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:09:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:09:53Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:09:53Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:53Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:53Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:53Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:53Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:53Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:09:53Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:09:53Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:13Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:10:13Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:13Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:13Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:13Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:13Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:13Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:13Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:13Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:13Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:33Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:10:33Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:33Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:33Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:33Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:33Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:33Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:33Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:33Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:33Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:53Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:10:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:10:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:10:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:10:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:11:14Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:14Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:14Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:14Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:14Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:14Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:14Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:14Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:14Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:11:34Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:34Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:34Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:34Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:34Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:34Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:34Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:34Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:34Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:11:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:11:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:11:54Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:11:54Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:11:54Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:11:54Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:14Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:12:14Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:14Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:14Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:14Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:14Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:14Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:14Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:14Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:14Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:34Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:12:34Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:34Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:34Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:34Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:34Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:34Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:34Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:34Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:34Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:54Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:54Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:54Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:54Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:12:55Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:12:55Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:55Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:55Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:55Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:55Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:12:55Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:55Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:12:55Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:13:15Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:15Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:15Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:15Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:15Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:15Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:15Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:15Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:15Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:35Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:13:35Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:35Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:35Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:35Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:35Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:35Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:35Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:35Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:35Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:13:55Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:13:55Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:13:55Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:13:55Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:13:55Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:13:55Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:14:15Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:15Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:15Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:15Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:15Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:15Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:15Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:15Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:15Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:15Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:14:16Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:16Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:16Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:16Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:14:16Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:16Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:16Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:16Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:16Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:16Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:14:36Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:36Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:36Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:36Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:36Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:36Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:36Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:36Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:36Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:56Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:14:56Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:14:56Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:56Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:56Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:56Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:56Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:14:56Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:56Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:14:56Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:16Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Applying mariaDB resources. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Finished applying Database Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Finished applying storage Resources {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Performing Database Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Performing Object Storage Health Check {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:15:16Z INFO Object Storage Health Check Failed {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:16Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:16Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:16Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:16Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:16Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:16Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:16Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:16Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:15:36Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:36Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:36Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:36Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:36Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:15:36Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:36Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:36Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:36Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-pipelines-prj-643148, this bundle will be included in external tls connections. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:41Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-pipelines-prj-643148 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:15:41Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:41Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:41Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:15:41Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:41Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO Encountered error when attempting to fetch ConfigMap: [openshift-service-ca.crt]. Error: &ConfigMap{ObjectMeta:{ 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Data:map[string]string{},BinaryData:map[string][]byte{},Immutable:nil,} {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:41Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", "error": "configmaps \"openshift-service-ca.crt\" not found"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:15:41Z INFO Publishing Ready Metrics {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:15:41Z INFO DatabaseAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO ObjectStoreAvailable {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO APIServerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO PersistenceAgentReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO ScheduledWorkflowReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO WorkflowControllerReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO MLMDProxyReady {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:15:41Z INFO Ready {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:15:41Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:15:41Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:15:41Z INFO DSPA resources cleaned up. {"namespace": "test-pipelines-prj-643148", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:44Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:46Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:46Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:46Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:46Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:46Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:46Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:46Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:46Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:46Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:48Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:48Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:48Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:48Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:48Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:48Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:48Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:48Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:48Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:48Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:49Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:16:49Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:16:49Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:49Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:49Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:49Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:16:49Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:49Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:49Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:16:49Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:17:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:17:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:17:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:17:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:17:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:17:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:17:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:18:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:01Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:01Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:01Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:01Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:18:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:18:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:18:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:18:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:18:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:19:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:01Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:01Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:01Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:01Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:19:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:19:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:19:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:19:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:19:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:20:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:01Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:01Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:01Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:01Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:20:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:20:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:20:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:20:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:20:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:21:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:21:22Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:22Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:22Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:22Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:22Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:22Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:22Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:22Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:22Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:42Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:21:42Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:21:42Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:42Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:42Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:21:42Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:42Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:42Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:42Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:21:42Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:22Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:22:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:22:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:22:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:22:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:23:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:23Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:23:23Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:23Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:23Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:23Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:23Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:23Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:23Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:23Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:23Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:43Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:23:43Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:23:43Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:43Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:43Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:43Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:43Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:23:43Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:43Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:23:43Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:03Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:03Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:03Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:03Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:04Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:04Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:04Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:04Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:04Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:04Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:24Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:24Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:24Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:24Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:24Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:24Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:24Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:24Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:24Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:24Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:44Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:44Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:44Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:44Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:44Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:44Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:44Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:44Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:47Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:47Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:47Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:47Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:47Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:47Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:47Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:47Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:47Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:47Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:47Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:24:47Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:24:47Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:24:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:24:59Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:59Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:59Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:24:59Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:24:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z ERROR Reconciler error {"controller": "datasciencepipelinesapplication", "controllerGroup": "datasciencepipelinesapplications.opendatahub.io", "controllerKind": "DataSciencePipelinesApplication", "DataSciencePipelinesApplication": {"name":"dspa","namespace":"test-dsp-schedule-prj-115465"}, "namespace": "test-dsp-schedule-prj-115465", "name": "dspa", "reconcileID": "030e6a73-b19e-42e4-aa5f-bf9d3099af48", "error": "Operation cannot be fulfilled on deployments.apps \"mariadb-dspa\": the object has been modified; please apply your changes to the latest version and try again"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:495 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:01Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:01Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:01Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:01Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:02Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:02Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:02Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:02Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:02Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:07Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:07Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:07Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:07Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:07Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:07Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:07Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:18Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:18Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:18Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:18Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:18Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:18Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:18Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:18Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:18Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:18Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:38Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:38Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:38Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:38Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:38Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:38Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:38Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:38Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:38Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:38Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:58Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:58Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:58Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:58Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:25:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:25:59Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:25:59Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:25:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:19Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:19Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:19Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:19Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:19Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:19Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:19Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:19Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:19Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:19Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:39Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:39Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:39Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:39Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:39Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:39Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:39Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:39Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:59Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:26:59Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:26:59Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:59Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:59Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:59Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:59Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:26:59Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:59Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:26:59Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:19Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:27:19Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:19Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:27:19Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:27:19Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:19Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:19Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:27:19Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:19Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:19Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:39Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:39Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:39Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:27:40Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:27:40Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:27:40Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:40Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:40Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:40Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:27:40Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:40Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:27:40Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:28:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:28:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:28:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:28:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:28:40Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:28:40Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:40Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:40Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:28:40Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:40Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:40Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:40Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:28:40Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:29:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:29:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:29:40Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:29:40Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:40Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:40Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:40Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:40Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:40Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:29:40Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:29:40Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:00Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:30:00Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:00Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:00Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:00Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:00Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:00Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:00Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:00Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:00Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:20Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:30:20Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:20Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:20Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:20Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:20Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:20Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:20Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:20Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:20Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:40Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:30:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:30:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:30:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:30:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:31:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:01Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:01Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:01Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:01Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:31:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:31:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:31:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:31:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:31:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:01Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:01Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:01Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:01Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:01Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:01Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:01Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:01Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:01Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:01Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:21Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:21Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:21Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:21Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:21Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:21Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:21Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:21Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:21Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:21Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:41Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO MLMD not specified, but is a required component for Pipelines. Including MLMD with default specs. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Secret [ds-pipeline-db-dspa] already exists, using stored value. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Applying mariaDB resources. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Finished applying Database Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Using externalStorage, bypassing object storage deployment. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Finished applying storage Resources {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Performing Database Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Encountered error when processing custom ca bundle, Error: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Unable to connect to Database: error parsing CA Certificate, ensure provided certs are in valid PEM format {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Performing Object Storage Health Check {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z ERROR encountered error when processing custom ca bundle or proxy configuration {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "error parsing CA Certificate, ensure provided certs are in valid PEM format"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.init.func2 /workspace/controllers/storage.go:164 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).isObjectStorageAccessible /workspace/controllers/storage.go:255 github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:346 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:41Z INFO Object Storage Health Check Failed {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Health check for Database or Object Store failed, retrying in 20 seconds. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:41Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:41Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:41Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:41Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:41Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:41Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:41Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:41Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:47Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:47Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:47Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Found global CA Bundle odh-trusted-ca-bundle present in this namespace test-dsp-schedule-prj-115465, this bundle will be included in external tls connections. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z ERROR Failed to extract or resolve DSPA parameters {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", "error": "configmaps \"dsp-trusted-ca-dspa\" is forbidden: unable to create new content in namespace test-dsp-schedule-prj-115465 because it is being terminated"} github.com/opendatahub-io/data-science-pipelines-operator/controllers.(*DSPAReconciler).Reconcile /workspace/controllers/dspipeline_controller.go:315 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:222 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:479 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:438 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func1.1 /cachi2/output/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.23.3/pkg/internal/controller/controller.go:313 2026-07-07T07:32:47Z INFO Publishing Ready Metrics {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO PersistenceAgentReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO ScheduledWorkflowReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO WorkflowControllerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO Ready {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO ObjectStoreAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO APIServerReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "False"} 2026-07-07T07:32:47Z INFO MLMDProxyReady {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO DatabaseAvailable {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa", " Status:": "Unknown"} 2026-07-07T07:32:47Z INFO No other DSPAs with PipelineStore 'kubernetes' found. Cleaning up webhook resources. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO Webhook resources cleanup complete. {"namespace": "redhat-ods-applications", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO DSPA resources cleaned up. {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} 2026-07-07T07:32:47Z INFO DataSciencePipelinesApplication.datasciencepipelinesapplications.opendatahub.io "dspa" not found {"namespace": "test-dsp-schedule-prj-115465", "dspa_name": "dspa"} ==================== WORKLOAD POD LOG: rhoai-model-registries/model-catalog-7cc8755dc9-wdzcm.log ==================== W0707 06:19:38.838336 1 deprecated.go:66] ==== Removed Flag Warning ====================== logtostderr is removed in the k8s upstream and has no effect any more. =============================================== I0707 06:19:38.838462 1 kube-rbac-proxy.go:553] Reading config file: /etc/kube-rbac-proxy/config-file.yaml I0707 06:19:38.839638 1 kube-rbac-proxy.go:247] Valid token audiences: I0707 06:19:38.840637 1 kube-rbac-proxy.go:370] Reading certificate files I0707 06:19:38.841077 1 kube-rbac-proxy.go:418] Starting TCP socket on 0.0.0.0:8443 I0707 06:19:38.841083 1 kube-rbac-proxy.go:466] Starting TCP socket on 0.0.0.0:8888 I0707 06:19:38.841504 1 kube-rbac-proxy.go:425] Listening securely on 0.0.0.0:8443 I0707 06:19:38.841517 1 kube-rbac-proxy.go:473] Listening securely on 0.0.0.0:8888 for proxy endpoints W0707 06:19:38.741008 1 connect.go:86] Retrying connection to PostgreSQL (attempt 1/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused W0707 06:19:39.743970 1 connect.go:86] Retrying connection to PostgreSQL (attempt 2/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused W0707 06:19:41.746245 1 connect.go:86] Retrying connection to PostgreSQL (attempt 3/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused W0707 06:19:44.750663 1 connect.go:86] Retrying connection to PostgreSQL (attempt 4/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused W0707 06:19:48.756117 1 connect.go:86] Retrying connection to PostgreSQL (attempt 5/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused W0707 06:19:53.762344 1 connect.go:86] Retrying connection to PostgreSQL (attempt 6/25): failed to connect to `user=catalog_user database=model_catalog`: 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused 172.30.21.245:5432 (model-catalog-postgres): dial error: dial tcp 172.30.21.245:5432: connect: connection refused I0707 06:19:59.780469 1 connect.go:94] Successfully connected to PostgreSQL database I0707 06:19:59.780512 1 catalog.go:121] Leader election configured: lock duration=1m0s, heartbeat=15s I0707 06:19:59.786068 1 service.go:130] Connecting to EmbedMD service... I0707 06:19:59.786091 1 service.go:137] Connected to EmbedMD service I0707 06:19:59.786096 1 repos.go:35] Getting types... I0707 06:19:59.786123 1 election.go:190] Attempting to acquire leadership... I0707 06:19:59.788831 1 election.go:226] Successfully acquired leadership lock: catalog-leader W0707 06:19:59.799209 1 service.go:186] Unable to get types: no types available: ERROR: relation "Type" does not exist (SQLSTATE 42P01) I0707 06:19:59.809095 1 service.go:162] Running migrations... I0707 06:19:59.998917 1 service.go:167] Migrations completed I0707 06:19:59.998956 1 service.go:169] Syncing types... I0707 06:20:00.060562 1 service.go:174] Syncing types completed W0707 06:20:00.060590 1 election.go:132] Leader callback 0 exited early I0707 06:20:00.799391 1 repos.go:35] Getting types... I0707 06:20:00.800705 1 repos.go:47] Types retrieved I0707 06:20:00.800728 1 repos.go:63] All required types validated successfully 2026/07/07 06:20:00 INFO initializing plugin plugin=mcp version=v1alpha1 basePath=/api/mcp_catalog/v1alpha1 2026/07/07 06:20:00 INFO initializing plugin plugin=model version=v1alpha1 basePath=/api/model_catalog/v1alpha1 I0707 06:20:00.801100 1 performance_metrics.go:154] Loading performance metrics data from [/shared-benchmark-data] I0707 06:20:00.805992 1 performance_metrics.go:239] Built model directory cache (72 models indexed) 2026/07/07 06:20:00 INFO all plugins initialized count=2 2026/07/07 06:20:00 INFO mounting plugin routes plugin=mcp 2026/07/07 06:20:00 INFO mounting plugin routes plugin=model 2026/07/07 06:20:00 INFO starting plugin plugin=mcp I0707 06:20:00.806124 1 base.go:49] Starting mcp plugin in read-only mode (standby) I0707 06:20:00.806187 1 loader.go:84] Initializing MCP loader - parsing configs W0707 06:20:00.807801 1 config.go:24] Configuration file /data/default-sources/sources.yaml uses deprecated 'catalogs' field. Please rename to 'model_catalogs'. I0707 06:20:00.807832 1 loader.go:149] reading MCP catalog config type yaml... I0707 06:20:00.807843 1 loader.go:165] loaded MCP source rh_mcp_servers of type yaml I0707 06:20:00.807846 1 loader.go:149] reading MCP catalog config type yaml... I0707 06:20:00.807850 1 loader.go:165] loaded MCP source rh_partner_mcp_servers of type yaml I0707 06:20:00.807853 1 loader.go:149] reading MCP catalog config type yaml... I0707 06:20:00.807856 1 loader.go:165] loaded MCP source community_mcp_servers of type yaml I0707 06:20:00.807963 1 loader.go:92] MCP loader config parsing complete I0707 06:20:00.807982 1 base.go:64] mcp plugin read-only mode initialized 2026/07/07 06:20:00 INFO starting plugin plugin=model I0707 06:20:00.808038 1 base.go:49] Starting model plugin in read-only mode (standby) W0707 06:20:00.808426 1 config.go:24] Configuration file /data/default-sources/sources.yaml uses deprecated 'catalogs' field. Please rename to 'model_catalogs'. I0707 06:20:00.808446 1 loader.go:204] reading config type yaml... I0707 06:20:00.808459 1 loader.go:222] loaded source redhat_ai_models of type yaml I0707 06:20:00.808462 1 loader.go:204] reading config type yaml... I0707 06:20:00.808465 1 loader.go:222] loaded source redhat_ai_validated_models of type yaml I0707 06:20:00.808468 1 loader.go:204] reading config type yaml... I0707 06:20:00.808472 1 loader.go:222] loaded source other_models of type yaml I0707 06:20:00.808584 1 base.go:64] model plugin read-only mode initialized I0707 06:20:00.808689 1 catalog.go:176] Catalog API server listening on 0.0.0.0:8080 I0707 06:20:00.808737 1 catalog.go:195] Became leader - notifying plugins 2026/07/07 06:20:00 INFO plugin becoming leader plugin=model I0707 06:20:00.808798 1 base.go:97] model plugin becoming leader 2026/07/07 06:20:00 INFO plugin becoming leader plugin=mcp I0707 06:20:00.808852 1 base.go:97] mcp plugin becoming leader I0707 06:20:00.808881 1 loader.go:101] MCP loader performing leader operations I0707 06:20:00.808897 1 loader.go:192] Loading MCP servers from source: Red Hat MCP Servers (id: rh_mcp_servers) I0707 06:20:00.811809 1 loader.go:379] Reading models from yaml source redhat_ai_models I0707 06:20:00.821772 1 loader.go:379] Reading models from yaml source redhat_ai_validated_models I0707 06:20:00.910384 1 loader.go:379] Reading models from yaml source other_models I0707 06:20:00.914691 1 base.go:112] model plugin leader mode active I0707 06:20:00.914700 1 loader.go:285] Loading model redhat_ai_models:RedHatAI/granite-3.3-8b-instruct with 1 artifact(s) I0707 06:20:00.949199 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Apertus-8B-Instruct-2509-FP8-dynamic with 1 artifact(s) I0707 06:20:01.134530 1 loader.go:192] Loading MCP servers from source: Red Hat Partner MCP Servers (id: rh_partner_mcp_servers) I0707 06:20:01.167057 1 performance_metrics.go:273] Loaded 117 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Apertus-8B-Instruct-2509-FP8-dynamic I0707 06:20:01.167083 1 loader.go:285] Loading model redhat_ai_models:RedHatAI/granite-4.0-h-small with 1 artifact(s) I0707 06:20:01.181408 1 loader.go:285] Loading model other_models:RedHatAI/Qwen3-Embedding-8B with 1 artifact(s) I0707 06:20:01.241959 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/DeepSeek-R1-0528-quantized.w4a16 with 1 artifact(s) I0707 06:20:01.353240 1 performance_metrics.go:273] Loaded 55 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/DeepSeek-R1-0528-quantized.w4a16 I0707 06:20:01.353273 1 loader.go:285] Loading model redhat_ai_models:RedHatAI/granite-4.0-h-tiny with 1 artifact(s) I0707 06:20:01.368874 1 loader.go:285] Loading model other_models:RedHatAI/all-MiniLM-L6-v2 with 1 artifact(s) I0707 06:20:01.384404 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Devstral-Small-2-24B-Instruct-2512 with 1 artifact(s) I0707 06:20:01.672769 1 performance_metrics.go:273] Loaded 138 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Devstral-Small-2-24B-Instruct-2512 I0707 06:20:01.672800 1 loader.go:285] Loading model redhat_ai_models:granite-7b-redhat-lab with 1 artifact(s) I0707 06:20:01.735542 1 loader.go:285] Loading model other_models:RedHatAI/embeddinggemma-300m with 1 artifact(s) I0707 06:20:01.753783 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Kimi-K2-Instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:01.776516 1 loader.go:192] Loading MCP servers from source: Community MCP Servers (id: community_mcp_servers) I0707 06:20:01.831432 1 performance_metrics.go:273] Loaded 22 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Kimi-K2-Instruct-quantized.w4a16 I0707 06:20:01.831464 1 loader.go:285] Loading model redhat_ai_models:granite-8b-starter-v1 with 1 artifact(s) I0707 06:20:01.844652 1 loader.go:285] Loading model other_models:RedHatAI/granite-embedding-english-r2 with 1 artifact(s) I0707 06:20:01.860660 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.1-8B-Instruct with 1 artifact(s) I0707 06:20:01.894399 1 loader.go:115] MCP loader leader operations complete I0707 06:20:01.894424 1 base.go:112] mcp plugin leader mode active I0707 06:20:02.052724 1 performance_metrics.go:273] Loaded 107 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.1-8B-Instruct I0707 06:20:02.052757 1 loader.go:285] Loading model redhat_ai_models:granite-8b-lab-v1 with 1 artifact(s) I0707 06:20:02.066500 1 loader.go:285] Loading model other_models:RedHatAI/nomic-embed-text-v1.5 with 1 artifact(s) I0707 06:20:02.066520 1 loader.go:420] other_models: loaded 5 models I0707 06:20:02.082527 1 loader.go:431] other_models: cleaned up 0 models I0707 06:20:02.086195 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF with 1 artifact(s) I0707 06:20:02.204612 1 performance_metrics.go:273] Loaded 30 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF I0707 06:20:02.204642 1 loader.go:285] Loading model redhat_ai_models:granite-3.1-8b-lab-v1 with 1 artifact(s) I0707 06:20:02.218296 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic with 1 artifact(s) I0707 06:20:02.292282 1 performance_metrics.go:273] Loaded 35 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.1-Nemotron-70B-Instruct-HF-FP8-dynamic I0707 06:20:02.292328 1 loader.go:285] Loading model redhat_ai_models:granite-8b-code-instruct with 1 artifact(s) I0707 06:20:02.306732 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct with 1 artifact(s) I0707 06:20:02.378942 1 performance_metrics.go:273] Loaded 35 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct I0707 06:20:02.378974 1 loader.go:285] Loading model redhat_ai_models:granite-8b-code-base with 1 artifact(s) I0707 06:20:02.394214 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic with 1 artifact(s) I0707 06:20:02.469600 1 performance_metrics.go:273] Loaded 36 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic I0707 06:20:02.469631 1 loader.go:285] Loading model redhat_ai_models:prometheus-8x7b-v2-0 with 1 artifact(s) I0707 06:20:02.469695 1 loader.go:420] redhat_ai_models: loaded 10 models I0707 06:20:02.485790 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:02.488826 1 loader.go:431] redhat_ai_models: cleaned up 0 models I0707 06:20:02.592924 1 performance_metrics.go:273] Loaded 30 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w4a16 I0707 06:20:02.592954 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w8a8 with 1 artifact(s) I0707 06:20:02.680266 1 performance_metrics.go:273] Loaded 43 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-3.3-70B-Instruct-quantized.w8a8 I0707 06:20:02.680297 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct with 1 artifact(s) I0707 06:20:02.725709 1 performance_metrics.go:273] Loaded 3 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct I0707 06:20:02.725736 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct-FP8 with 1 artifact(s) I0707 06:20:02.794255 1 performance_metrics.go:273] Loaded 18 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Maverick-17B-128E-Instruct-FP8 I0707 06:20:02.794348 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct with 1 artifact(s) I0707 06:20:02.853368 1 performance_metrics.go:273] Loaded 15 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct I0707 06:20:02.853406 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic with 1 artifact(s) I0707 06:20:02.935465 1 performance_metrics.go:273] Loaded 40 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-FP8-dynamic I0707 06:20:02.935511 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:03.001503 1 performance_metrics.go:273] Loaded 16 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Llama-4-Scout-17B-16E-Instruct-quantized.w4a16 I0707 06:20:03.001543 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic with 1 artifact(s) I0707 06:20:03.055425 1 performance_metrics.go:273] Loaded 3 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic I0707 06:20:03.055457 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:03.080475 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 with 1 artifact(s) I0707 06:20:03.101342 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/MiniMax-M2.5 with 1 artifact(s) I0707 06:20:03.183721 1 performance_metrics.go:273] Loaded 41 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/MiniMax-M2.5 I0707 06:20:03.183751 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Ministral-3-14B-Instruct-2512 with 1 artifact(s) I0707 06:20:03.366271 1 performance_metrics.go:273] Loaded 130 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Ministral-3-14B-Instruct-2512 I0707 06:20:03.366299 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Ministral-3-3B-Instruct-2512 with 1 artifact(s) I0707 06:20:03.574380 1 performance_metrics.go:273] Loaded 149 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Ministral-3-3B-Instruct-2512 I0707 06:20:03.574413 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512 with 1 artifact(s) I0707 06:20:03.641460 1 performance_metrics.go:273] Loaded 18 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512 I0707 06:20:03.641487 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512-NVFP4 with 1 artifact(s) I0707 06:20:03.729613 1 performance_metrics.go:273] Loaded 42 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Large-3-675B-Instruct-2512-NVFP4 I0707 06:20:03.729657 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501 with 1 artifact(s) I0707 06:20:03.848248 1 performance_metrics.go:273] Loaded 81 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501 I0707 06:20:03.848277 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501-FP8-dynamic with 1 artifact(s) I0707 06:20:03.863302 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501-quantized.w4a16 with 1 artifact(s) I0707 06:20:03.882474 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-24B-Instruct-2501-quantized.w8a8 with 1 artifact(s) I0707 06:20:03.898685 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503 with 1 artifact(s) I0707 06:20:04.063929 1 performance_metrics.go:273] Loaded 105 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503 I0707 06:20:04.063961 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-FP8-dynamic with 1 artifact(s) I0707 06:20:04.272196 1 performance_metrics.go:273] Loaded 112 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-FP8-dynamic I0707 06:20:04.272233 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w4a16 with 1 artifact(s) I0707 06:20:04.458303 1 performance_metrics.go:273] Loaded 104 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w4a16 I0707 06:20:04.458334 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w8a8 with 1 artifact(s) I0707 06:20:04.670353 1 performance_metrics.go:273] Loaded 123 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-3.1-24B-Instruct-2503-quantized.w8a8 I0707 06:20:04.670408 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603 with 1 artifact(s) I0707 06:20:04.828875 1 performance_metrics.go:273] Loaded 40 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603 I0707 06:20:04.828910 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603-NVFP4 with 1 artifact(s) I0707 06:20:04.966815 1 performance_metrics.go:273] Loaded 91 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mistral-Small-4-119B-2603-NVFP4 I0707 06:20:04.966918 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Mixtral-8x7B-Instruct-v0.1 with 1 artifact(s) I0707 06:20:05.077421 1 performance_metrics.go:273] Loaded 59 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Mixtral-8x7B-Instruct-v0.1 I0707 06:20:05.077450 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 with 1 artifact(s) I0707 06:20:05.284691 1 performance_metrics.go:273] Loaded 126 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 I0707 06:20:05.284727 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 with 1 artifact(s) I0707 06:20:05.455391 1 performance_metrics.go:273] Loaded 80 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 I0707 06:20:05.455484 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 with 1 artifact(s) I0707 06:20:05.588967 1 performance_metrics.go:273] Loaded 79 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 I0707 06:20:05.588998 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 with 1 artifact(s) I0707 06:20:05.703533 1 performance_metrics.go:273] Loaded 31 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 I0707 06:20:05.703566 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamic with 1 artifact(s) I0707 06:20:05.792563 1 performance_metrics.go:273] Loaded 46 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/NVIDIA-Nemotron-Nano-9B-v2-FP8-dynamic I0707 06:20:05.792595 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Phi-4-mini-instruct-FP8-dynamic with 1 artifact(s) I0707 06:20:05.965243 1 performance_metrics.go:273] Loaded 110 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Phi-4-mini-instruct-FP8-dynamic I0707 06:20:05.965274 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Phi-4-reasoning-FP8-dynamic with 1 artifact(s) I0707 06:20:06.140407 1 performance_metrics.go:273] Loaded 75 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Phi-4-reasoning-FP8-dynamic I0707 06:20:06.140446 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct with 1 artifact(s) I0707 06:20:06.333426 1 performance_metrics.go:273] Loaded 102 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct I0707 06:20:06.333477 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-FP8-dynamic with 1 artifact(s) I0707 06:20:06.491029 1 performance_metrics.go:273] Loaded 110 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-FP8-dynamic I0707 06:20:06.491066 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:06.667952 1 performance_metrics.go:273] Loaded 87 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w4a16 I0707 06:20:06.667986 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8 with 1 artifact(s) I0707 06:20:06.860728 1 performance_metrics.go:273] Loaded 106 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen2.5-7B-Instruct-quantized.w8a8 I0707 06:20:06.860758 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-8B-FP8-dynamic with 1 artifact(s) I0707 06:20:07.133395 1 performance_metrics.go:273] Loaded 155 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-8B-FP8-dynamic I0707 06:20:07.133431 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-480B-A35B-Instruct-FP8 with 1 artifact(s) I0707 06:20:07.188539 1 performance_metrics.go:273] Loaded 13 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-480B-A35B-Instruct-FP8 I0707 06:20:07.188565 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-Next-NVFP4 with 1 artifact(s) I0707 06:20:07.371593 1 performance_metrics.go:273] Loaded 119 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-Coder-Next-NVFP4 I0707 06:20:07.371624 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-Next-80B-A3B-Instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:07.588293 1 performance_metrics.go:273] Loaded 133 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-Next-80B-A3B-Instruct-quantized.w4a16 I0707 06:20:07.588326 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3-VL-235B-A22B-Instruct-NVFP4 with 1 artifact(s) I0707 06:20:07.759237 1 performance_metrics.go:273] Loaded 79 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3-VL-235B-A22B-Instruct-NVFP4 I0707 06:20:07.759265 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3.5-122B-A10B-FP8-dynamic with 1 artifact(s) I0707 06:20:07.859142 1 performance_metrics.go:273] Loaded 42 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3.5-122B-A10B-FP8-dynamic I0707 06:20:07.859178 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3.5-35B-A3B-FP8-dynamic with 1 artifact(s) I0707 06:20:08.051548 1 performance_metrics.go:273] Loaded 108 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3.5-35B-A3B-FP8-dynamic I0707 06:20:08.051580 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic with 1 artifact(s) I0707 06:20:08.134212 1 performance_metrics.go:273] Loaded 25 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/Qwen3.5-397B-A17B-FP8-dynamic I0707 06:20:08.134251 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/Voxtral-Mini-3B-2507-FP8-dynamic with 1 artifact(s) I0707 06:20:08.155201 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gemma-2-9b-it with 1 artifact(s) I0707 06:20:08.173956 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gemma-2-9b-it-FP8 with 1 artifact(s) I0707 06:20:08.195803 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gemma-3n-E4B-it-FP8-dynamic with 1 artifact(s) I0707 06:20:08.349286 1 performance_metrics.go:273] Loaded 81 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gemma-3n-E4B-it-FP8-dynamic I0707 06:20:08.349386 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-120b with 1 artifact(s) I0707 06:20:08.587150 1 performance_metrics.go:273] Loaded 148 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-120b I0707 06:20:08.587189 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-120b-essential with 1 artifact(s) I0707 06:20:08.934537 1 performance_metrics.go:273] Loaded 148 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-120b-essential I0707 06:20:08.934582 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-20b with 1 artifact(s) I0707 06:20:09.228856 1 performance_metrics.go:273] Loaded 177 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-20b I0707 06:20:09.228893 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/gpt-oss-20b-essential with 1 artifact(s) I0707 06:20:09.528917 1 performance_metrics.go:273] Loaded 177 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/gpt-oss-20b-essential I0707 06:20:09.528956 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-base-quantized.w4a16 with 1 artifact(s) I0707 06:20:09.549612 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct with 2 artifact(s) I0707 06:20:09.733307 1 performance_metrics.go:273] Loaded 95 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct I0707 06:20:09.733343 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-FP8-dynamic with 1 artifact(s) I0707 06:20:09.783753 1 performance_metrics.go:273] Loaded 6 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-FP8-dynamic I0707 06:20:09.783784 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w4a16 with 1 artifact(s) I0707 06:20:09.837143 1 performance_metrics.go:273] Loaded 4 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w4a16 I0707 06:20:09.837182 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w8a8 with 1 artifact(s) I0707 06:20:09.877662 1 performance_metrics.go:273] Loaded 1 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-3.1-8b-instruct-quantized.w8a8 I0707 06:20:09.877692 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-4.0-h-small-FP8-dynamic with 1 artifact(s) I0707 06:20:09.958873 1 performance_metrics.go:273] Loaded 42 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-4.0-h-small-FP8-dynamic I0707 06:20:09.958900 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/granite-4.0-h-tiny-FP8-dynamic with 1 artifact(s) I0707 06:20:10.067593 1 performance_metrics.go:273] Loaded 46 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/granite-4.0-h-tiny-FP8-dynamic I0707 06:20:10.067622 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4 with 1 artifact(s) I0707 06:20:10.182624 1 performance_metrics.go:273] Loaded 55 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4 I0707 06:20:10.182663 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4-FP8-dynamic with 1 artifact(s) I0707 06:20:10.331082 1 performance_metrics.go:273] Loaded 54 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4-FP8-dynamic I0707 06:20:10.331125 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w4a16 with 1 artifact(s) I0707 06:20:10.436398 1 performance_metrics.go:273] Loaded 52 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w4a16 I0707 06:20:10.436432 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w8a8 with 1 artifact(s) I0707 06:20:10.557793 1 performance_metrics.go:273] Loaded 60 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/phi-4-quantized.w8a8 I0707 06:20:10.557823 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/sarvam-105b-FP8-dynamic with 1 artifact(s) I0707 06:20:10.674026 1 performance_metrics.go:273] Loaded 61 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/sarvam-105b-FP8-dynamic I0707 06:20:10.674056 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/sarvam-30b-FP8-dynamic with 1 artifact(s) I0707 06:20:10.868632 1 performance_metrics.go:273] Loaded 105 performance metrics artifacts for model redhat_ai_validated_models:RedHatAI/sarvam-30b-FP8-dynamic I0707 06:20:10.868672 1 loader.go:285] Loading model redhat_ai_validated_models:RedHatAI/whisper-large-v3-turbo-quantized.w4a16 with 1 artifact(s) I0707 06:20:10.868734 1 loader.go:420] redhat_ai_validated_models: loaded 76 models I0707 06:20:10.962197 1 loader.go:431] redhat_ai_validated_models: cleaned up 0 models 2026/07/07 06:30:47 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:43678 - 200 472B in 171.93µs 2026/07/07 06:30:47 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:43658 - 200 420B in 6.571442ms 2026/07/07 06:30:47 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:43674 - 200 9130B in 6.320997ms 2026/07/07 06:30:48 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:43658 - 200 47545B in 6.396033ms 2026/07/07 06:30:48 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:43678 - 200 141549B in 8.412457ms 2026/07/07 06:30:48 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:43692 - 200 84064B in 22.522576ms 2026/07/07 06:30:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43678 - 200 41125B in 42.996783ms 2026/07/07 06:30:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43658 - 200 56B in 4.341597ms 2026/07/07 06:30:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43678 - 200 62309B in 46.113477ms 2026/07/07 06:30:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:39056 - 200 20726B in 50.620846ms 2026/07/07 06:31:02 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic HTTP/1.1" from 127.0.0.1:43678 - 200 16070B in 2.407825ms 2026/07/07 06:31:02 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:43658 - 200 32975B in 9.974177ms 2026/07/07 06:31:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:43658 - 200 32975B in 8.831691ms 2026/07/07 06:31:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=variant_group_id.string_value%3D%22c359df52-82cb-4d43-be32-6dafea427522%22&pageSize=10&source=redhat_ai_validated_models HTTP/1.1" from 127.0.0.1:43678 - 200 16124B in 3.191703ms 2026/07/07 06:31:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&pageSize=20&recommendations=true&targetRPS=1 HTTP/1.1" from 127.0.0.1:43658 - 200 41124B in 32.304159ms 2026/07/07 06:31:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90&pageSize=20&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43658 - 200 41124B in 27.495109ms 2026/07/07 06:31:44 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:43658 - 200 420B in 1.572731ms 2026/07/07 06:31:44 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:43658 - 200 472B in 53.478µs 2026/07/07 06:31:44 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:43658 - 200 9130B in 3.879816ms 2026/07/07 06:31:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:43658 - 200 141549B in 8.103485ms 2026/07/07 06:31:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:43678 - 200 47545B in 4.399482ms 2026/07/07 06:31:45 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:43658 - 200 84064B in 5.080839ms 2026/07/07 06:31:46 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43678 - 200 20726B in 67.560483ms 2026/07/07 06:31:46 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46482 - 200 56B in 65.616762ms 2026/07/07 06:31:46 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43658 - 200 62309B in 86.46192ms 2026/07/07 06:31:46 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46468 - 200 41125B in 98.223048ms 2026/07/07 06:32:00 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic HTTP/1.1" from 127.0.0.1:46482 - 200 16070B in 2.718943ms 2026/07/07 06:32:00 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:43678 - 200 32975B in 11.274982ms 2026/07/07 06:32:01 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts HTTP/1.1" from 127.0.0.1:43678 - 200 32975B in 7.792165ms 2026/07/07 06:32:01 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=variant_group_id.string_value%3D%22c359df52-82cb-4d43-be32-6dafea427522%22&pageSize=10&source=redhat_ai_validated_models HTTP/1.1" from 127.0.0.1:43678 - 200 16124B in 3.354704ms 2026/07/07 06:32:01 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&pageSize=20&recommendations=true&targetRPS=1 HTTP/1.1" from 127.0.0.1:43678 - 200 41124B in 32.252251ms 2026/07/07 06:32:02 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90&pageSize=20&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43678 - 200 41124B in 30.879259ms 2026/07/07 06:36:03 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36008 - 200 420B in 2.870539ms 2026/07/07 06:36:10 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36008 - 200 420B in 1.588932ms 2026/07/07 06:36:27 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36008 - 200 420B in 1.613427ms 2026/07/07 06:36:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:36008 - 200 472B in 87.139µs 2026/07/07 06:36:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36008 - 200 420B in 1.585072ms 2026/07/07 06:36:50 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:36008 - 200 9130B in 3.879212ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:36008 - 200 47545B in 4.866399ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:58674 - 200 141549B in 8.749963ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:58676 - 200 84064B in 23.922152ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58696 - 200 56B in 66.626878ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 20726B in 85.370333ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58674 - 200 41125B in 94.505954ms 2026/07/07 06:36:51 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58698 - 200 62309B in 113.684619ms 2026/07/07 06:37:10 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36008 - 200 420B in 1.585106ms 2026/07/07 06:37:10 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:36008 - 200 9130B in 1.913185ms 2026/07/07 06:37:10 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:58696 - 200 472B in 48.534µs 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:58696 - 200 141549B in 7.988418ms 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:58696 - 200 47545B in 3.870907ms 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:58696 - 200 84064B in 4.280988ms 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58696 - 200 56B in 5.82144ms 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58696 - 200 20726B in 61.107265ms 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 62309B in 80.319735ms 2026/07/07 06:37:11 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:44970 - 200 41125B in 89.027127ms 2026/07/07 06:37:40 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:36008 - 200 472B in 79.846µs 2026/07/07 06:37:40 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:58696 - 200 420B in 2.096884ms 2026/07/07 06:37:40 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:58696 - 200 9130B in 2.309713ms 2026/07/07 06:37:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:36008 - 200 47545B in 6.217077ms 2026/07/07 06:37:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:58696 - 200 141549B in 8.599292ms 2026/07/07 06:37:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:58202 - 200 84064B in 18.858672ms 2026/07/07 06:37:43 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58220 - 200 56B in 21.121523ms 2026/07/07 06:37:43 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 41125B in 59.367275ms 2026/07/07 06:37:43 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58696 - 200 62309B in 69.255388ms 2026/07/07 06:37:43 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58224 - 200 20726B in 73.043646ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=artifacts.use_case.string_value%3D%27chatbot%27+AND+artifacts.ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=artifacts.ttft_p90.double_value&pageSize=10&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 292781B in 1.683574452s 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-3B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 29027B in 45.614546ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-14B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58220 - 200 59712B in 55.125081ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b-essential/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43386 - 200 52679B in 149.054036ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43372 - 200 52497B in 239.680991ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fsarvam-30b-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43392 - 200 24427B in 184.706738ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 62309B in 270.999525ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:58220 - 200 41125B in 287.31542ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen2.5-7B-Instruct-quantized.w8a8/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43386 - 200 35997B in 180.897773ms 2026/07/07 06:37:56 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen3-8B-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43392 - 200 37135B in 101.605788ms 2026/07/07 06:37:57 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FLlama-3.1-8B-Instruct/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 55242B in 91.354702ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:36008 - 200 420B in 2.949835ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:36008 - 200 472B in 55.47µs 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:36008 - 200 9130B in 3.538527ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:43372 - 200 47545B in 4.714784ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:36008 - 200 84064B in 6.961582ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:43372 - 200 141549B in 7.076953ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43372 - 200 62309B in 68.791047ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:35734 - 200 56B in 53.764975ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:36008 - 200 20726B in 65.02855ms 2026/07/07 06:38:16 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:35732 - 200 41125B in 80.860497ms 2026/07/07 06:38:29 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?filterQuery=artifacts.use_case.string_value%3D%27chatbot%27+AND+artifacts.ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=artifacts.ttft_p90.double_value&pageSize=10&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43372 - 200 292781B in 1.45294107s 2026/07/07 06:38:29 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-3B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43372 - 200 29027B in 314.192138ms 2026/07/07 06:38:29 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46154 - 200 41125B in 289.09949ms 2026/07/07 06:38:29 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FMinistral-3-14B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:35734 - 200 59712B in 394.980797ms 2026/07/07 06:38:29 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46134 - 200 62309B in 387.46565ms 2026/07/07 06:38:30 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b-essential/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46144 - 200 52679B in 481.399941ms 2026/07/07 06:38:30 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgpt-oss-20b/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46130 - 200 52497B in 505.832994ms 2026/07/07 06:38:30 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fsarvam-30b-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46154 - 200 24427B in 152.354288ms 2026/07/07 06:38:30 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen2.5-7B-Instruct-quantized.w8a8/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:43372 - 200 35997B in 246.601108ms 2026/07/07 06:38:30 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FQwen3-8B-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46134 - 200 37135B in 251.994065ms 2026/07/07 06:38:30 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FLlama-3.1-8B-Instruct/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:35734 - 200 55242B in 243.231412ms 2026/07/07 06:38:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/labels HTTP/1.1" from 127.0.0.1:46130 - 200 472B in 153.851µs 2026/07/07 06:38:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources HTTP/1.1" from 127.0.0.1:46144 - 200 420B in 2.770585ms 2026/07/07 06:38:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models/filter_options HTTP/1.1" from 127.0.0.1:55636 - 200 9130B in 4.342309ms 2026/07/07 06:38:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI HTTP/1.1" from 127.0.0.1:46144 - 200 47545B in 5.127129ms 2026/07/07 06:38:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=null HTTP/1.1" from 127.0.0.1:46144 - 200 84064B in 5.227872ms 2026/07/07 06:38:41 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/models?pageSize=4&sourceLabel=Red+Hat+AI+validated HTTP/1.1" from 127.0.0.1:46130 - 200 141549B in 11.221598ms 2026/07/07 06:38:42 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2Fgemma-2-9b-it/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:55656 - 200 56B in 48.692472ms 2026/07/07 06:38:42 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDeepSeek-R1-0528-quantized.w4a16/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46144 - 200 20726B in 52.514374ms 2026/07/07 06:38:42 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FApertus-8B-Instruct-2509-FP8-dynamic/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:46130 - 200 41125B in 64.068848ms 2026/07/07 06:38:42 "GET http://rh-ai.apps.ods-qe-psi-07.osp.rh-ods.com/api/model_catalog/v1alpha1/sources/redhat_ai_validated_models/models/RedHatAI%2FDevstral-Small-2-24B-Instruct-2512/artifacts/performance?filterQuery=use_case.string_value%3D%27chatbot%27+AND+ttft_p90.double_value+%3C%3D+2231.15348815918&latencyProperty=ttft_p90&orderBy=ttft_p90.double_value&pageSize=999&recommendations=true&sortOrder=ASC&targetRPS=1 HTTP/1.1" from 127.0.0.1:55654 - 200 62309B in 88.661886ms ==================== WORKLOAD POD LOG: rhoai-model-registries/model-catalog-postgres-686d85b578-k7ttz.log ==================== The files belonging to this database system will be owned by user "postgres". This user must also own the server process. The database cluster will be initialized with locale "en_US.utf8". The default database encoding has accordingly been set to "UTF8". The default text search configuration will be set to "english". Data page checksums are disabled. fixing permissions on existing directory /var/lib/pgsql/data/userdata ... ok creating subdirectories ... ok selecting dynamic shared memory implementation ... posix selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting default time zone ... UTC creating configuration files ... ok running bootstrap script ... ok performing post-bootstrap initialization ... ok syncing data to disk ... ok Success. You can now start the database server using: pg_ctl -D /var/lib/pgsql/data/userdata -l logfile start initdb: warning: enabling "trust" authentication for local connections initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. waiting for server to start....2026-07-07 06:19:45.080 UTC [30] LOG: redirecting log output to logging collector process 2026-07-07 06:19:45.080 UTC [30] HINT: Future log output will appear in directory "log". done server started /var/run/postgresql:5432 - accepting connections => sourcing /usr/share/container-scripts/postgresql/start/set_passwords.sh ... ALTER ROLE waiting for server to shut down.... done server stopped Starting server... 2026-07-07 06:19:45.481 UTC [1] LOG: redirecting log output to logging collector process 2026-07-07 06:19:45.481 UTC [1] HINT: Future log output will appear in directory "log".